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ABSTRACT 


The Defense Logistics Agency Disposition Services (DLA DS) is facing substantial 
potential budget cuts, which will require the organization to increase efficiency and cut 
costs. DLA DS is assessing the feasibility to improve the item disposal process with an 
object/shape recognition technology as a means to increase efficiency. The technology 
may potentially reduce the time required to identify items in order to route for appropriate 
disposal. With the advancement of new technology and the availability of databases, it 
may now be possible to identify an object using recognition technology. Given the 
potential of these technologies to reduce identification time and errors, research to assess 
the viability of existing prototypes is justified. This applied project identified and 
assessed object/shape recognition technologies as a means to support efficiency while 
taking into account wider government policy and objectives in relation to technology 
adoption. This project evaluated the feasibility of implementation of these technologies 
that includes an analysis of DLA DS’s items disposal process, a technology readiness 
assessment of the technology, and a cost-benefit analysis to assess the financial payoffs 
of an investment in the technology. The objective was to understand the items disposal 
process and assess object/shape recognition technology to address the problem that 
unlabeled items poses to DLA DS at Camp Pendleton, CA. 
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EXECUTIVE SUMMARY 


The Defense Logistics Agency Disposition Services (DLA DS) is facing substantial 
potential budget cuts, which will require the organization to increase efficiency and cut 
costs. DLA DS conducted a Lean Six Sigma study in 2008 that led to streamlining the 
item disposal process and saving time equivalent to 40% of a full time employee. The 
Lean Six Sigma study concluded with a recommendation to identify and study the 
potential of technologies to support further efficiency gains (DLA DS, 2008). 

This master’s thesis was motivated by that recommendation. In the past decade, 
several companies have developed object/shape recognition systems to meet specific 
commercial requirements, with some success (Rajpurohit et ah, 2013). The objective of 
this study was to identify and assess the viability of object recognition technologies for 
use by DLA DS. 

This study is a mixed-method design including analysis of qualitative and 
quantitative data. Specifically, this study includes an analysis of DLA DS’s items 
disposal process, a technology readiness assessment (TRA) of object/shape recognition 
technology, and a cost-benefit analysis (CBA) to assess the financial payoffs of an 
investment in the technology. The TRAs and CBA are consistent with the Circular A 94, 
DOD Directive 5000.2 Interim and DOD TRA guidance (0MB, 1992; DOD, 2013a; 
ASD[R&E], 2011). The study assessed a four-month period of data and utilized a market 
comparable approach to calculate the benefits and costs and return on investment. As a 
public service sector organization, DLA DS does not generate revenues. The study used 
the market comparable labor to estimate revenues. 

This thesis’ objective was to understand the items disposal process and assess 
object/shape recognition technology to address the problem that unlabeled items poses to 
DLA DS at Camp Pendleton, CA. However, the findings generated by the analysis of 
DLA DS’s items disposal processing show three other areas of concern that contributed 
to days’ worth of backlogged items. The data analysis suggests that the problem is not 
exactly the unlabeled items. Indeed, employees spend excess time conducting research on 
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items that are not labeled or are missing DD 1348 forms. However, unlabeled items are 
fewer than 1% of accepted items, and handling them does not solely ’create a backlog. 
The following three other areas of concern also contributed to days’ worth of backlog. 
First, improperly filled DD 1348 provides for additional time spent conducting research 
in order to fill the DD 1348 form for further processing. Second, the receipt control 
numbers (RCNs) are not limited to the amount of items that can arrive in a pallet or tri¬ 
box. A RCN can have one or 50 items; therefore on average the facility accepts 425 items 
per day. The DLA DS at Camp Pendleton’s capacity to process items is limited to 300 
items with six-employee crew. Therefore, third, DLA DS at Camp Pendleton accepts 
more items than what it can process. The facility processes an average of 300 items per 
day; however, the excess items that were not processed contributed to have an average of 
47 RCN equaling to about 2,259 items or 7.5 days’ worth of backlogged items stored for 
later processing. 

Furthermore, the thesis examined the alternative of addressing the unlabeled items 
issues with the use of new object/shape recognition technology. The DLA DS 
technological requirement includes item recognition and disposal information feedback, 
and the ability to take pictures to use on DLA email system to communicate and upload 
the picture to liquidations website. The TRA for three commercial-off-the-shelf 
technologies shows that they do not meet the maturity standard of TRL 7 required for 
DOD projects. However, Imaginestics’ technology is the most viable and mature system, 
ranking up to TRL 5. Imaginestics’ technology can be implemented in less considerable 
time and cost in reference to the other two companies. 

The results of the CBA shows a total quantified net economic benefit occurring 
from the technology’s contributions to be an estimated $65,000 corresponding to an 
internal rate of return of 15% over a 10-year time span and profitability index (PI) 1.43, 
indicating that for every dollar invested $1.43 in benefits is accrued with the payback 
period of almost seven years. The new technology will save money in labor costs for the 
unlabeled items; however, it only address less than 1% of items and not the 7.5 days’ 
worth of backlogged items for the DLA DS facility located in Camp Pendleton. 
Therefore, the results from the queuing theory suggests that with nine employees the 
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DLA DS at Camp Pendleton will keep up with demand by processing all received items 
and will progressively reduce the 7.5 days’ worth of backlogged items. 

Areas of uncertainty remain for future study. This study was not able to capture 
time spent in every step of the item disposal process, or observe whether the backlog is a 
result of specific events such as retrograding units from deployments or the time of the 
year. The study did not assess the potential added value lost of scrapped items that might 
have been sold. For example if the technology increased the items sent for liquidation 
there would be an increase in revenues to DLA. Additionally, data was collected at one 
site, the Camp Pendleton DLA DS facility, such that the results may not be indicative of 
all DLA DS. 

One of the main conclusions resulting from this thesis effort is that researching 
items is not the only area of concern with regard to process inefficiency. Rather, this 
thesis identified three other areas of concern suggested by the data analysis and 
observation. Therefore, further recommendations were provided to take care of the other 
three problems. 

The TRA and CBA show that the implementation of object/shape recognition is 
viable, however the technology will address delays associated with the less than 1% of 
the backlogged items at Camp Pendleton. This study recommends DLA DS take steps to 
address additional identified concerns. This study recommends that DLA DS: 

1. Making mandatory to fill the DD 1348 on DLA DS website, and 

2. Limiting the amount of items on pallets to match the facility’s processing 
capacity, and 

3. Increasing the Camp Pendleton facility to nine employees, or 

4. Adding a night shift such that the shift only process items without 
accepting or conducting mundane tasks, or 

5. Once a month stop accepting items for disposal process for a full week in 
order to focus the effort to dispose the backlog items, or 
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6. 


Fill the gap with technology by implement full automation with optical 
sorting and data mining that included sensors, laser, object/shape 
recognition technology on conveyor belt system and data analytics to 
improve materiel’s opportunity to be sold as commodity, or liquidations, 
and improve operational efficiency; which would be consistent with DLA 
Director’s “Big Idea” and DLA DS’s strategic technological goal to 
accomplish. 

The technological innovation disrupts status quo by ensuring a plan of increased 
capacity for the long run and perhaps reduces some of the over $60 million of labor cost 
that account for about 35% of DLA’s budget. 
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I. 


INTRODUCTION 


A. OVERVIEW 

The budget cuts the Department of Defense (DOD) proposes to implement are 
creating a spending habit shift of the Defense Logistics Agency Disposition Services 
(DLA DS). The budget cuts will require the organization to increase efficiency and cut 
costs. DLA DS conducted a Lean Six Sigma study in 2008 that led to streamlining the 
item disposal process and saving time equivalent to 40% of a full time employee. The 
Lean Six Sigma study concluded with a recommendation to identify and study the 
potential of technologies to support further efficiency gains (DLA DS, 2008). 

This master’s thesis was motivated by that recommendation. In the past decade, 
several companies have developed object/shape recognition systems to meet specific 
commercial requirements, with some success (Rajpurohit et ah, 2013). The objective of 
this study was to identify and assess the viability of object recognition technologies for 
use by DLA DS to increase efficiency and reduce costs. This chapter describes the 
problem facing DLA DS, which motivated this study; details the research questions that 
guided the study; provides a description of DLA DS technology requirements; and 
outlines the organization of this report. 

B. THE PROBLEM STATEMENT 

According to the Government Accountability Office (GAO, 2010), the Defense 
Logistic Agency (DLA) Dispositions Services (DS) identified $1 billion worth of 
materiel for reutilization or disposal between fiscal years (FY) 2006 and 2008. However, 
identifying some materiel to allow reutilization or disposal has been difficult to 
efficiently accomplish. Time spent researching to identify material could be a key factor 
driving backlogs at DLA DS. Additionally, the research process may result in 
inefficiency and decrease goal attainment. In 2008, DLA DS initiated steps to improve 
the identification process by improving process flow, but further opportunities for 
improvement might be accomplished with technology (DLA DS, 2008). 
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In general, successful operations enhancement with technology saves money by 
providing faster service, reducing backlogs and reducing the cost of storing items. 
However, implementing technology has associated costs, including the cost of acquiring 
a technology that meets DLA requirements and also the cost associated with developing a 
technology for the specific needs of DLA DS. Development of immature technology 
increases the investment cost thereby increasing the payback period and reduces the 
willingness to invest. An effective evaluation of the technology’s maturity is required to 
support informed decisions. While the DLA DS item disposal process flow has been 
improved, additional efficiency improvements are desired (DLA DS, 2014a). This 
applied research assesses the potential of object/shape recognition technology to improve 
the DLA DS’s item disposal process by facilitating item identification. The aim is to 
evaluate the current state of object/shape recognition technology and assess the feasibility 
of implementing it at DLA DS. 

C. RESEARCH QUESTIONS, SCOPE AND LIMITATIONS OF THE STUDY 

This study addresses the questions that follow. 

1. Is object/shape recognition technology a viable tool to improve DLA DS 
operations? 

2. What are the technology needs for DLA DS? 

3. What technologies are available and are they mature enough for 
implementation? 

4. How much will the technology cost and will it generate any tangible 
savings? 

5. What is the payback period? 

I. Scope and Limitations of the Study 

This study is a mixed-method design including analysis of qualitative and 
quantitative data. Specifically this study includes an analysis of DLA DS’ items disposal 
process, a technology readiness assessment (TRA) of object/shape recognition 
technology, and a cost benefit analysis (CBA) to assess the financial payoffs of an 
investment in the technology. The TRAs and CBA are consistent with the Circular A-94, 
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DOD Directive 5000.2 Interim and DOD TRA guidance (0MB, 1992; DOD, 2013a; 
ASD[R&E], 2011). The study assessed a four-month period of data and utilized a market 
comparable approach to calculate the benefits and costs, and return on investment. As a 
public service sector organization, DLA DS does not generate revenues. The study used 
the market comparable labor to estimate revenues. 

D. DLA DISPOSITION SERVICES TECHNOLOGY REQUIREMENTS 

In recent years, significant progress has been made in the development of 
object/shape recognition technologies and their use in several real-time vision systems for 
detecting specific classes of objects within complex environments. This study assess the 
potential of several companies’ technologies to meet the four technological needs that the 
DLA DS requires: 

1. Identify government property via object/shape recognition or barcodes 
with national stock numbers (NSNs). 

2. Provide disposition instructions. 

3. Meet cyber security standards and information assurance requirement. 

4. Take pictures to use on the DLA email system to communicate or upload 
to liquidation website. 

E. ORGANIZATION OF STUDY 

The remainder of this report is organized as follows. Chapter II reviews of 
relevant academic research including, technology commercialization, technology 
transition and technology readiness assessment research as well as literature from the 
DOD to provide a background and framework for this study. Chapter III describes the 
method used to investigate the research questions. Chapter IV describes the results of the 
analysis of Disposition Services at Camp Pendleton’s items disposal process and the TRA 
of three companies’ prototypes. Chapter V describes the CBA for object/shape 
recognition available technologies and discusses the results. Finally, chapter VI 
summarizes the key findings of the study, details recommendations for DLA DS 
supported by the findings, and makes recommendations for future study. 
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II. LITERATURE REVIEW AND BACKGROUND 


A. OVERVIEW 

This chapter reviews academic literature and U.S. government policies in order to 
provide a description of the known understanding and developments in this area. The 
decision to adopt technology, particularly by government departments, is not an easy 
process and is not simply technology assessment in isolation. Many factors and processes 
must be taken into account. The objective of the chapter is to provide an overview of 
concepts and terminology related to government procurement of commercial items. This 
chapter reviews (1) the academic literature related to technology commercialization (TC) 
including challenges of developing and commercializing technology, (2) studies 
describing the generally accepted approach for cost-benefit analysis (CBA), and, (3) 
policies, procedures, and guidance of the United States’ statute and law related to the 
acquisition of commercial-off-the-shelf (COTS) products, technology transition, and 
technology readiness assessment. This review provides a background and suggests a 
method to identify, evaluate and develop a solution to the DLA DS’s property 
identification problem. 

B. UNDERSTANDING THE TECHNOLOGY COMMERCIALIZATION 
PROCESS 

New technologies have great potential but their commercialization has often 
proven challenging (Zahra & Nielsen, 2002). Technology Commercialization (TC) 
is of interest to aeademics and industry, and it has thus received a great deal of attention 
in the literature. Research has focused on identifying the processes to successfully 
manage commercialization; the development and launch of new products (Jolly, 1997, 
p. 20). TC seeks to identify the process through which innovative ideas are transformed 
into marketable products and to suggest means of overcoming challenges to 
commercialization. 
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1. What Is Technology Commercialization 

TC is defined as the “process of acquiring ideas, augmenting them with 
complementary knowledge, developing and manufacturing saleable goods, and selling 
the goods in the market” (Zahra & Nielsen, 2002). Additionally, Jolly (1997) explained 
TC as “taking a design through development and then manufacture and market it” (p. 25). 
The literature on TC suggests that successful technology commercialization processes act 
as a technology accelerator (Markman, Siegel, & Wright, 2008). TC involves a series of 
incremental steps to move a technology along the technology-readiness scale. As 
explained by Jolly (1997), bringing new technologies to market requires five activities: 
imagining, incubating, demonstrating, promoting, and adopting. Mobilizing stakeholders, 
resources, and constituents, and delivering the product bridges the TC activities. The 
steps of the TC process should provide a means to assess whether the technology will 
work in a real application scenario as well as to evaluate the technology’s ease of use and 
determine how useful it will be and whether it is likely to be sustainable in the market 
(Jolly, 1997). If successful, TC will eventually introduce new technology to the 
marketplace and increase a firm’s competitive advantage (Zahra & Nielsen, 2002). 
However, TC has proven challenging (Jolly, 1997; Zahra & Nielsen, 2002). 

2. Technology Commercialization Challenges 

TC requires strategic decision-making in an uncertain environment and often 
outlaying upfront a large amount of capital. This creates challenges for stakeholders, 
which often may prove too great for to overcome. One of the fundamental reasons TC 
fails is due to having limited knowledge of what the customer wants or needs and the 
technology’s acceptance rate in the marketplace. According to Jolly (1997), failure is due 
to the demand never materializing or perception of poor product quality. Poor quality 
results in a lack of stakeholder interest and support, because the innovation doesn’t live 
up to the promised capacity (Jolly, 1997). Investors may perceive that the technology is 
too risky or unlikely to result in a worthwhile benefit, and a lack of funding may prevent 
research and development (R&D) and the commercialization process (Jolly, 1997). For 
example, in the DOD the process of budgeting for innovation has many constraints that 
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often make it difficult to continue to fund research and development (R&D) along the 
complete pipeline of development, consequently stalling projects for months (DOD, 
2003). This hiatus is known as the “valley of death.” However, according Jolly (Jolly, 
1997), it is important to look at the big picture and figure out the links of failure or 
success in order to prevent further failure to continue to maximize success. 

3. What Makes Technology Commercialization Work? 

Assuming genuine innovation and strong market research, consistent 
communication across stakeholders and investor funding along with controlling other 
basic traits can fundamentally strengthen the TC process. According Jolly (1997), 
“successful innovators tend to exchange more information and establish more efficient 
external communication links than their less successful counterparts” (p. 229). By taking 
this approach network uncertainty is naturally reduced through planning and increased 
communication among stakeholders. Good information is another important factor. In 
order to gather information during the TC process other scientists are considered the first 
customers of new technology (Jolly, 1997). They provide useful feedback to ensure that 
technological adjustments meet marketplace needs. Close management of the project 
during the early stages of product design refines the innovation process by monitoring 
and measuring resources used from R&D to marketing (Jolly, 1997). Additionally, 
incorporating market needs with the unique selling points to communicate how the new 
product is different and offers new functions increases acceptance (Jolly, 1997). Investor 
attraction requires a mix of the scope of the innovation in addition to the business and 
economical benefit rather than explaining the technological benefit alone (Kaarela, 2013). 
These actions support the successful commercialization of technology, thus boosting the 
confidence of stakeholders and potential investors (Figure 1). 
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the Process of Technology Commerciauzation 


SUBPROCESSES: BUILDING THE VALUE OF A NEW TtCHNOLOOV 








I 


3, 


IMAGINING /X INCUBATING 
lh«Du«l / \ toDefin* 

CTeehno- / NOomomci*!-, 

Mwket) / 2. \ iMbiUy 

limght I Mobihzing 
Inltiett tnd 
Endotiemml 


y 


EMONSTRATINj 
Conttxliulty 
In Prodiutt 

^ \ And / 6 

Mobiliiing I Mobibzmg 

R«fouic«f 1 M»dcel 

fot f \ Conitdufntf 

Demontti*- 
Uon 


PROMOTING 
adoption 


SUSTAINING 

Conusetcitlixtiion 


8 

Mobihzing 1 
IComplenoDt-j 
\t:yAsMtifo^ 
Dcbvniy , 







BRIDGES: SATISFYING AND MOBIUZING STAKEHOLDERS AT EACH STAGE 


Figure 1. Technology Commercialization Process (from Jolly, 1997) 


The TC process posed by Jolly (1997) illustrates a methodical approach to 
commercializing technology, as shown in Figure 1. According to Jolly (1997), 
stakeholders’ interest, recognition, and moving resources are key principles of 
innovation. Stakeholder interest includes nine interconnected stages. Five subprocesses 
build value into the new technology by “imagining a techno-market insight; incubating 
the technology to define its commercializability; demonstrating it contextually in 
products and/or processes; promoting the latter’s adoption; and sustaining 
commercialization use” (Jolly, 1997, p. 46). Four additional stages bridge the 
subprocesses to satisfy and mobilize stakeholders and resources along the stages. The 
following five subprocesses are described: 

1. Imagining: idea stage, linked together the incubating stage with 
“mobilizing interest and endorsement” (Jolly, 1997, p. 48). 

2. Incubating: the “defining moment” where the resources are allocated 
(Jolly, 1997, p. 54); the demonstration stage is linked with the mobilizing 
resources stage. 

3. Demonstration: product development links together the promoting stage 
with mobilizing market stage (Jolly, 1997, p. 60) 
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4. 


Promoting: “persuading people to adopt” (Jolly, 1997, p. 65) links 
together the sustaining stage with mobilizing assets for delivery. 

5. Sustaining: “long presence in the market that generates fair shares on the 
long run” (Jolly, 1997, p. 69). 

In summary, without recognition that all stages are equally important in the 
process—meaning that, without stakeholders’ interest, appropriate action and moving 
valuable resources in between every stage the commercialization—technology often fails. 

This section discussed Jolly’s (1977) model as a tool in the TC. Communication 
and information gathering reduces uncertainty and gains the interest of the investors, who 
then provide project funding. Investors care about the bottom line and about showing the 
financial benefit that will have to be adopted in order to illustrate the financial cost and 
benefit of the project. Additionally, knowing enough of what the marketplace wants and 
needs is a function of successful TC. Finally, incorporating Jolly’s (1977) model provides 
guidance to facilitate the management of product development. TC is successful when 
challenges are overcome by including shareholders and investors in the communication 
process during every stage of commercialization. 

C. COST-BENEFIT ANALYSIS 

A cost-benefit analysis (CBA) is a tool that managers use to assess a project’s net 
benefit in monetary terms (Boardman et ah, 2011) During the system’s engineering phase 
of the development process the DOD Directive (DODD) 5000.2 Interim, claims a CBA 
needs to be conducted, which resolves the “...supportability analysis, provides insight 
into supportability drivers and includes the impact of resources on readiness supported by 
engineering analyses required for product support” (DOD, 2013a, p. 215). In other words, 
all options must be considered and alternatives must be analyzed to showcase which one 
provides a greater impact and solution to meet user needs. 

In the United States government, the guidance to conduct a CBA for any federal 
program is directed by the Circular A-94 Guidelines and Discount Rates for Benefit-Cost 
Analysis of Federal Programs (0MB, 1992). The Circular A-94 (0MB, 1992) provides 
instructional steps on how to plan and pursue a cost effectiveness analysis (CEA) in order 
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to depict the risk associated with implementing a project and lists the better options that 
out-weigh benefit over risk. 

D. TECHNOLOGY TRANSITION 

The DOD relies on having technologically innovative weapons systems in order 
to have an upper hand over any adversary. The technology transition (TT) process is a 
prominently accepted method by both private and public sectors to create effective 
technology systems. Congress, the DOD, and universities have established a number of 
“technology transition” programs (DOD, 2003; GAO, 2005). There are many variations 
of TT since technology transition has been practiced from the early 1800s (Geels, 2001). 
TT by itself is not the answer to successful product launch; however, it paves the way to 
smoothly and efficiently transition along the evolutionary acquisition system (DOD, 
2003). The Defense Advanced Research Projects Agency (DARPA) and the U.S. Air 
Force (USAF) are examples of organizations that have fully embraced TT processes to 
innovate highly technological weapons. 

1. What Is Technology Transition? 

Technology transition occurs when an innovative concept is developed into a 
technological tool the R&D phase then transferred to the marketplace for use (Markman 
et ah, 2008). The USAF (2010) defines technology transition in a more detailed “in- 
house” process that includes the following activities: “establishing a team, formulating 
the plan, developing information, coordinating and updating the information, obtaining 
commitment and approval at the proper stages, and executing transition.” In other words, 
the DARPA (2010) refers to technology transition as getting technology from research 
and into use. 

2. What Are Technology Transition Challenges? 

Technology transfers is a process that is prone to failure. One reason for TT’s 
failure is lack of oversight and supervision. According to the GAO (2005), “failure to 
track even the most basic information, such as the number of projects completed, could 
result in a lack of ability to manage the program properly and poor stewardship of 
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taxpayer money” (p. 25). This oversight contributes to instability of project progress to 
success and project failure. The GAO (2005) conducted a study on TT programs. The 
study centered on the management process, oversight and assessment practices for TT 
programs. The research highlights many of the ongoing barriers: 

At this time, however, the transition programs have limited measures to 
gauge individual project success and program impact or return on 
investment in the long term. At best, they are collecting after action 
reports that describe the results of transition projects, and occasionally 
identify some cost savings, but not in a consistent manner. In addition, 
there are inconsistencies in how the reports are being prepared, reviewed, 
and used. The Quick Reaction Fund program manager, in fact, had trouble 
just getting projects to submit after action reports. (GAO, 2005, p. 22) 

3. What Makes Technology Transition Work? 

Consistent supervision can fundamentally impact a smooth TT process. The GAO 
(2005) research suggested the following: 

Selecting promising projects for funding is not enough to ensure 
successful transition. Program managers must also actively oversee 
implementation to make sure that project goals are being met and the 
program is working as intended and to identify potential barriers to 
transition. They must also sustain commitment from acquirers. Moreover, 
the transition program as a whole must have good visibility over progress 
and be positioned to shift attention and resources to problems as they 
arise, (p. 18) 

One of the best approach that supports TT success is to establish a database that 
facilitates the collection of pertinent information (GAO, 2005). The TT process posed by 
the USAF (2010) illustrates a methodical approach to technology transition as shown in 
Figure 2. The model highlights the linear process from accepting or approving a concept 
up to transitioning to a prototype. 
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Figure 2. Technology Transition Process Flow Diagram (from USAF, 2010) 


The USAF Technology Transition model in Figure 2 illustrates phases and stage- 
gates that are significant to the smooth process of TT and is applicable to this research. 
The figure illustrates a method to measure and approve technology before moving to the 
next stage. Although this model is depicted in a linear fashion the innovation process is 
often a circular model with many cycles of proof of concept and development before a 
prototype is ready. The TT process published by the USAF (2010) illustrates a 
managerial approach to approve a concept and transition technology from lab to a 
weapon system. The USAF adopted the stage-gate process due to its “industry-proven 
mechanism for effectively managing, directing, [and] otherwise controlling new product 
development efforts” (USAF, 2010, p. 8). The explanation of the activities and stage- 
gates is as follows: 

1. Establish a team: Establish a team of stakeholders, managers and 
gatekeepers in order to make appropriate strategic decisions during all 
phases of technology transition. 

2. Eormulate: The team conducts an assessment using the Technology 
Readiness Eevel (TRE) tool to establish a baseline of the technology’s 
maturity and formulates a strategy to mature it for a final product. Section 
E provides TRE definitions and details. 


12 























3. Develop information: The team gathers milestone maturity information for 
approval to move to the next level. 

4. Coordinate and update: The team communicates across stakeholders and 
controls the approval process to move to the next gate. 

5. Commit and approve: The team evaluates and agrees to one of the 
decision criteria, “go/hold/no-go,” to transition the technology. 

6. Transition: The likelihood of successful technology transitions to produce 
a product. 

The six gate-stage phases are “intended to provide a disciplined approach to 
executing a technology maturation effort, including consideration of all technical and 
acquisition factors associated with a successful transition. Ultimately, the use of the 
stage-gate process will facilitate laboratory, engineering, and programmatic reviews” 
(USAF, 2010, p. 9). During the Proof of Concept stage, the team approves the concept to 
make a prototype. In the Refinement stage, the prototype is refined for Development in 
stage-gate 4. Subsequently, in stage-gate 5, the developed prototype is tested in a 
Relevant Environment and approved to transition to stage-gate 6. The USAF (2010) 
claimed that this method formalizes the process and keeps their “eye on the prize.” In 
general, this method is a building strategy, beginning with R&D to mature the technology 
and progress from Technology Readiness Level (TRL) 3 through TRL 6, then the project 
transitions to an acquisition program for further maturation (USAF, 2010, p. 8). 

The USAF’s Technology Transition model illustrates a revolutionary process to 
effectively and efficiently implement a concept into a system. This section described the 
managerial process to effectively transition technologies in the DOD. The next section 
explains how one moves from processes to TRLs. 

E. TECHNOLOGY READINESS LEVEL 

The DOD spends billions of dollars developing advanced major defense 
programs. Technology Readiness Assessment (TRA) helps with the evaluation of any 
given’s technology maturity in order to reduce risk and cost of implementation and 
utilization (Department of Energy [DOE], 2012). The TRA is a widely accepted model 
used in the U.S. government’s acquisition process. TRE is just one of two approaches for 
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assessing technology maturity. The GAO added an additional element to the evaluation 
process when assessing technology maturity. In addition to the TRA, the GAO (2003) 
presented a system’s profile element during the maturity assessment. The profile 
includes, a general description, pictures, key elements of the system, and a schedule that 
covers milestone and completion dates. Acquisitions of systems are not limited only to 
the DOD: in 2012, the Department of Energy (DOE) had over 400 active projects such 
that the DOE relied on the TRA methodology to review and assess these advance energy 
systems. Regardless of the agency or departments’ assessments requirement, the TRA 
and system profile both evaluate the technology in order to timely recognize and rectify 
gaps, mitigate risk, control timelines, and reduce or control cost. 

1. What Is Technology Readiness Assessment, and How Is it Measured? 

The DOD (2011) defined TRA as “a systematic, metric-based process that 
assesses the maturity of, and the risk associated with, critical technologies to be used in 
Major Defense Acquisition Programs (MDAPs)” (p. 1-1). In order to make the transition 
from the laboratory to product, the TRA can serve as a helpful knowledge-based standard 
and shorthand used to evaluate the technology’s maturity (ASD[R&E], 2011). The 
evaluation provides a numeric value know as Technology Readiness Eevel (TRE) that 
measures technology from mature or near mature. 

TRE is a disciplined ranking system of technology’s maturity rated between TRE 
1 and 9. The TRE analytical tool is well defined and is the best practice approach in 
evaluating the technology’s maturity. The milestone levels are as follow: TRE 1 contains 
conceptual documented paper studies of technology’s basic properties; TRE 2 is when the 
concept becomes a practical application to invent the system; TRE 3 is when the 
analytical and experimental laboratory scale models are observed; TRE 4 undertakes the 
component for validation in the laboratory; TRE 5 contains applied research laboratory 
scale experimentation; TRE 6 is where a model or prototype is demonstrated in a relevant 
environment; however, this is not the only consideration because the evaluation is 
supplemented with expert professional judgment; subsequently, TRE 7, is a prototype 
that can be presented and tested in actual relevant environment; TRE 8 is where the 
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actual system is tested to for demonstration in a plant environment, to consequently the 
systems is built in TRL 9, which is ready for use in operational mission conditions 
(ASD[R&E], 2011). (All the TRLs are more fully explained below in Table 1.) The 
DOD’s (2011) benchmark is whether technological evolution gives greater capabilities; 
however, to reap the benefit and not waste money during the R&D process, evaluating 
and assessing the technology’s maturity would increase efficiency of the investment 
(Sohn & Moon, 2003). 

Moreover, the GAO (2003) assessed systems in two components: (1) a system’s 
profile and (2) a product knowledge assessment: 

1. System profile: the first component is the system profile detailing the 
general description of the product with pictures and key elements such as 
schedule timeline and cost (GAO, 2003). 

2. Product knowledge assessment: the second component is the product 
knowledge assessment where the teehnology is assesses and given a TRL 
ranking (GAO, 2003). 

Knowing the levels provides an unbiased approach when evaluating technology 
maturity. According to the GAO (2003; 2009a) and DOE (2012), the benchmark and best 
practice to claim technology maturity is TRL 7. Significantly, TER 7 enables the 
technology prototype to be evaluated in a realistic operational environment resulting in 
low risk reduction to enter production and implementation. The GAO (2003; 2009a) 
stated that a system in TRL 6 is referred as approaching or nearing maturity and is 50% 
of the desire knowledge level. In a relevant environment, technology is evaluated in a 
high fidelity laboratory simulated operational environment, whereas an operational 
environment is where the system is intended to operate (ASD[R&E], 2011). As depicted 
in Eigure 3, the researcher drew from the DOE’s (2012) framework for the model flow of 
technology assessment progress in order to have a visual guide. Eurthermore, an 
illustration is provided in Table 1 that contains details of definition, description and 
supporting information of all the technology readiness levels in the TRL model. 
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Figure 3. Progress Flow of Technology Readiness Levels (from DOE, 2012) 


The doe’s (2012) TRE model was adapted to develop the guiding framework for 
this study as illustrated in Eigure 3. The process begins from research department such as 
a university or a research industry. Universities and industry research departments 
continuously provide input throughout the cycle but the model does not have to include 
this collaborative knowledge exchange format and is equally relevant to in-house 
development programs. Eevels of readiness increase as concepts are moved to laboratory 
for research. Once a prototype is developed, the prototype is tested in a mission 
environment where the technology is evaluated. Eull production occurs when TRE 9 is 
accomplished. At each stage/level feedback is imperative provided by the project’s 
manager and the project’s authority approving level in order to control the process and 
meet requirements. 

Adopted from the USAF Technology Development and Transition Strategy 
(TDTS) Guidebook (USAE, 2010), Table 1 provides a visual chart of definitions for the 
TRE from 1-9. Row 1 displays the numeric attributes for the technology readiness level, 
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row 2 displays the basic definition and row 3 displays a detailed description for each 
level. But, most importantly, a technology is considered fully mature at TRL 7, when the 
prototype is tested in the relevant operational environment and this level is the DOD’s 
minimum maturity level for consideration to satisfy requirements is TRL 6 (ASD[R&E], 
2011 ). 


Table 1. Technology Readiness Level Definitions (from USAF, 2010) 


TRL 

Definition 

Description 

1 

Basic Principles Observed and 
Reported 

Fewest level of technology readiness. Scientific 
research begins translation to applied research and 
development. Examples might include paper 
studies of a technology’s basic properties. 

2 

Technology Concept and/or 
Application Formulated 

Invention begins. Once basic principles are 
observed, practical applications can be invented. 
The application is speculative and there is no proof 
or detailed analysis to support the assumption. 
Examples remain limited to paper studies. 

3 

Analytical and Experimental 
Critical Function and/or 
Characteristic Proof of Concept 

Active research and development is initiated. This 
includes analytical studies and laboratory studies 
to physically validate analytical predictions of 
separate elements of the technology. Examples 
include components that are not yet integrated or 
representative. 

4 

Component and/or Breadboard 
Validated in Faboratory 
Environment 

Basic technological components are integrated to 
establish the feasibility that the pieces will work 
together. This is relatively low fidelity compared 
to the eventual system. Examples include 
integration of ad hoc hardware in a laboratory. 

5 

Component and/or Breadboard 
Validated in Relevant 
Environment 

Fidelity of breadboard technology increases 
significantly. The basic technological components 
are integrated with reasonably realistic supporting 
elements so that the technology can be tested in a 
simulated environment. Examples include high- 
fidelity laboratory integration of components. 

TRL 

Definition 

Description 


17 




6 

System/Subsystem Model or 
Prototype Demonstrated in 
Relevant Environment 

Representative model or prototype system well 
beyond the breadboard tested for TRE 5 undergoes 
test in a relevant environment. Represents a major 
step in a technology’s demonstrated readiness. 
Examples include prototype testing in a high- 
fidelity laboratory environment or simulated 
operational environment. 

7 

System Prototype Demonstrated 
in Operational Environment 

Prototype near or at planned operational system. 
Represents a major advance from TRE 6 and 
requires demonstration of an actual system 
prototype in an operational environment (e.g., in 
an aircraft or other vehicle or in space). Examples 
include prototype testing in a test bed aircraft. 

8 

Actual System Completed and 
Elight-Qualified Through Test 
and Demonstration 

Technology has been proven to work in its final 
form and under expected conditions. In almost all 
cases, this TRE represents the end of true system 
development. Examples include developmental test 
and evaluation of the system in its intended 
weapon system to determine whether it meets 
design specifications. 

9 

Actual System Elight-Proven 
Through Successful Mission 
Operations 

Actual application of the technology in its final 
form and under mission conditions, such as those 
encountered in operational test and evaluation. In 
almost all cases, this ends final debugging aspects 
of true system development. Examples include 
system usage under operational mission 
conditions. 


In summary, this section provided a general discussion of the label system when 
ranking technology’s maturity ranging between TRLs 1-9. Since the creation of the 
technology government, companies have used it increase competiveness by having 
policies in place to achieve successful products and systems development. The next 
section discusses what the U.S. government and the DOD have done to guide innovation. 

F. FEDERAL ACQUISITION STATUE AND LAW, AND DOD DIRECTIVE 
AND INSTRUCTION 

Over the past three decades, the U.S. government has developed an increased 
interest in regulating all acquisitions of supplies and services by all the federal executive 
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agencies. In addition, the federal government issued laws to regulate the technology 
transfer from the commercial sector to use in the DOD. This section reviews the Federal 
Acquisition Regulation (FAR), major DOD policies and directives related to regulating 
technology transition, and the risks and benefits associated with the acquisitions of 
commercial off-the-shelf items (COTS) to support the mission. 

1. Federal Acquisition Regulation 

U.S. government’s acquisition regulation became effective in April 1984 (FAR, 
2004). The government’s preference for procurement of commercial items is contained in 
the following statutes: 41 U.S.C. § 1906, § 1907, and § 3307 and 10 U.S.C. § 2375-2377 
(FAR, 2004, p. 12.1-1). In addition, the acquisition approach for the technology 
maturation effort of systems is covered in: 10 U.S.C. § 2304, 10 U.S.C. § 2305, and 10 
U.S.C. § 2306) (USAF, 2010, p25). The FAR (2004) is the strategic policy that directs 
agencies to “conduct market research to determine whether commercial items or non- 
developmental items are available that could meet the agency’s requirements” (p. 12.1- 
1). The foundation to link the private sector’s technological leadership with the DOD’s 
developmental technological needs was emphasized in the 2001 Quadrennial Defense 
Review Report in order to provide the best piece support in the field (OSD, 2001). The 
FAR is the documented source that guides DOD towards integration of the commercial 
sector’s capabilities to meet the U.S. military’s technological needs, while the DOD 
directives and instructions provide the template to managing and acquiring COTS. 

2. Department of Defense Directives 

The DOD annually spends billions of taxpayer dollars developing and procuring 
systems. In order to ensure positive returns on investment, the DOD has established 
policies and directives to adopt the “best practice” in developing and procuring 
equipment and weapons quickly and cheaply (GAO, 2003) The DOD’s best practice to 
manage the procurement of systems and technology transition procedures are underlined 
by the mandatory principles, DOD Directive (DODD) 5000.1, The Defense Acquisition 
System; and DOD Instruction (DODI) 5000.2 Interim, The Operation of the Defense 
Acquisition Systems. 
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a. 


DOD Directive 5000.1 


Congress and DOD leaders recognize the contribution and benefit associated with 
the procurement of COTS products, services, and technology for greater effectiveness 
and efficiency. One of the primary objectives of DODD 5000.1 (DOD, 2007) is to 
promote innovation, flexibility and management. The policy’s objective is “to acquire 
quality products that satisfy user needs with measurable improvements to mission 
capability and operational support, in a timely manner, and at a fair and reasonable price” 
(DOD, 2007, p. 3). The DODD 5000.1 (DOD, 2007) stated the following with regard to 
market research and procurement: 

DOD Component(s) shall consider multiple concepts and analyze possible 
alternative ways to satisfy the user need. ...The DOD Components shall 
seek the most cost-effective solution over the system’s life cycle. They 
shall conduct market research and analysis to determine the availability, 
suitability, operational supportability, interoperability, safety, and ease of 
integration of the considered and selected procurement solutions. The 
DOD Components shall work with users to define capability needs that 
facilitate the following, listed in descending order of preference: 

The procurement or modification of commercially available products, 
services, and technologies, from domestic or international sources, or the 
development of dual-use technologies, (p. 8) 

DODD 5000.1, The Defense Acquisition System, describes the management principles for 
DOD’s knowledge-based acquisition programs and these principles are a mandated 
directive that stresses best practice approach during the development of systems process 
(GAO, 2003). 

b. DOD Instruction 5000.2 Interim 

An important task during the conduct of technology transition is the enhancement 
or rapid development of military systems. The primary element involved during the 
system engineering processes is to “mature, prototype, and demonstrate the suite of 
selected system elements and complete the preliminary design of the full system for low- 
risk entry to Engineering and Manufacturing Development” (DOD, 2013a, p. 232). To 
promote the technology transition and maturation the DOD Instruction 5000.2 Interim 

(DOD, 2013a) provides the following amplifying guidance: 
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...an investment decision to pursue specific product or design concepts, 
and to commit the resources required to mature technology and/or reduce 
any risks that must be mitigated prior to decisions committing the 
resources needed for development leading to production and fielding. 

(p. 7) 

Furthermore, the directive provides detailed guidance with regard to acquiring COTS 
from the commercial sector such as small business, academia and government 
laboratories: 

Promising technologies shall be identified from all sources domestic and 
foreign, including government laboratories and centers, academia, and the 
commercial sector. In addition, PMs [program managers] shall consider 
the use of technologies developed under the Small Business Innovation 
Research (SBIR) program, and give favorable consideration to successful 
SBIR technologies. The risk of introducing these technologies into the 
acquisition process shall be reduced; coordination, cooperation, and 
mutual understanding of technology issues shall be promoted. The conduct 
of Science and Technology (S&T) activities shall not preclude, and where 
practicable, shall facilitate future competition. (DOD, 2013a, p. 14) 

DOD Instruction 5000.2 Interim, The Operation of the Defense Acquisition System, 
outlines a framework for managing acquisition programs. According the GAO (2013), 
about 40% of all 2012 systems were 25% or more over budget with over two years of 
scheduled delays. To provide control and improvement of these two elements—cost and 
schedule—the updated DODD is expected to provide flexibility to procure COTS. The 
directive indicates, with great emphasis, that there is a relationship between requirements, 
acquisitions, and budgeting processes such that all three components work together to 
identify and resolve issues early as possible (DOD, 2013a). Moreover, the directive 
suggests that actions required for cost reduction, especially for information technology 
systems, is achieved during the compliance to “redesign the processes that the system 
supports to reduce costs, improve effectiveness and maximize the use of commercial off- 
the-shelf technology” (DOD, 2013a, p. 71). 

c. Commercial-Off-the-Shelf Regulations 

The FAR and DOD regulation encourages the acquisitions of commercial-off-the 
shelf items. According to Allen (2008), the use of COTS involves accepting some 
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benefits and risks. However, the according to the DOD Instruction 5000.2, the use of 
COTS is worth investigating: 


The use of commercial-off-the-shelf (COTS) items, including Non- 
Developmental Items, can provide significant opportunities for 
efficiencies during system development but also can introduce certain 
issues that should be considered and mitigated if the program is to realize 
the expected benefits. (DOD, 2013a, p. 175) 

According to the directive (DOD, 2013a) the benefits associated with COTS include the 
following: 

1. System development time reduction 

2. A faster integration of technology 

3. Lower system life-cycle cost is lower due to leveraging available 
commercial technology 

Nevertheless, there are risks associated with using COTS. Allen (2008) put it best 
when specifically referring to the procuring of software COTS: stating “the risk is that 
selecting the wrong COTS software component might be more expensive than fixing 
problems in custom-built software” (p. 80). Furthermore, according to the directive 
(DOD, 2013a), there are concerns associated with COTS. The following, relevant to this 
study, are five major elements to name just a few a from wide-ranging list: 

1. “If integration requires a ‘modified COTS product,’ meaning that a COTS 
product may not be designed for many military environments (which, by 
definition, is not a COTS product under section 403 of title 41, United 
States Code, but is allowed under section 431 of title 41, United States 
Code), then the program may lose the ability to use the vendor’s 
subsequent product upgrades or to find a suitable replacement for the 
product from other commercial sources” (DOD, 2013a, p. 175). 

2. “The vendors can embed proprietary functions into COTS, limiting supply 
sources” (DOD, 2013a, p. 175). 

3. Supply chain risk management of COTS items is limited by the vendor, 
who is under no obligation to the purchaser to provide such information 
(DOD, 2013a, p. 175). 

4. Difficulty in finding suitable replacements and/or alternate items if the 
COTS vendor stops manufacturing the product or changes the 
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configuration drastically, requiring the need to maintain different 
configurations of a single product (DOD, 2013a, p. 175). 

5. The graphical user interface (GUI) design may not completely support 
user tasks, which can cause inefficient workarounds and improper use of 
the system by the user (DOD, 2013a, p. 175). 

According to Allen (2008), the best approach to reducing risk and concern when 
utilizing COTS is to adhere to DOD policies, maintain flexibility when incorporating 
COTS into the project, avoid restrictive statements of work and avoid very specific 
details that the technology ends up being unique to government (p. 80). Regardless of the 
extent to which the agency plans to use COTS, rigorous testing, evaluation, and 
acceptance of some risk must be undertaken in order to integrate the technology, 
streamline process, and reduce cost. 

This section introduced statutory preferences, directives and regulations that 
authorize the DOD to consider procurement of technological capabilities from the 
commercial sector. The core mission is for the DOD to consider commercial 
technological capabilities during the decision processes to possibly reduce development 
time, cost and leverage technology to sustain a competitive edge. Today, these laws and 
guidance dominate the DOD’s nature of acquiring major weapon systems in order to 
reduce cost (Markman et ah, 2008) 

G. COTS OBJECT/SHAPE RECOGNITION TECHNOLOGY 

In day-to-day routine, humans rely heavily on their vision to identify the world as 
they see it. This is a natural and effortless human process; however, to develop a machine 
to accomplish the process is a complex and ambitious project. Currently, a group of 
scientists has developed a prototype of object/shape recognition and images processing 
prototype that can identify leafs (Rajpurohit, Bhat, Devurkar, Patil, & Sirbi, 2013). 
Scientists are rapidly improving and advancing the technology and some major 
companies have adopted early versions. This section describes object/shape recognition 
technology and its utility in the private and government sector. 
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1. What Is Object/shape Recognition Technology 

Object/shape recognition technology is a tool that identifies items using a vision¬ 
capturing device and recognizes an object as a member of a category containing other 
similar objects (Frome, 1996). The application of the object/shape recognition technique 
by an image-capturing tool is an approach to identify items for the purpose of 
classification (Rajpurohit et ah, 2013). There are different approaches and methods, but 
all have elements in common: algorithm program, image capture apparatus and database 
to compare the image of interest to provide feedback. 

Object/shape recognition technology uses a program to index a database in order 
to translate received data, and then compares it to data in the database to provide 
feedback. In general, the infrastructure consists of several components that include an 
image capture apparatus for shape searching. The image signature is turned into digital 
data in order to compare potential leads in the database. The database server contains all 
information for matching as well as adding and classifying newly captured data. Another 
added feature would be that the program learns while comparing, so its information is 
readily available for future use. Finally, the tool returns feedback through wireless or 
cable connection. Conceptually, the tool identifies the object and its shape; then it 
translates that image into algorithm to digitally search a like category from the database; 
and finally, it provides feedback with the matching item (David & DeMenthon, 2006). 

2. Object/shape Recognition Technology’s Utility in Private and 
Government Sectors 

Many major companies in the private sector and U.S. government are using some 
form of object/shape recognition technology to differentiate and carve out a competitive 
edge, for instance, Amazon, with its newly released Fire Phone that has an application 
called Firefly. The Firefly application captures images of items for the purpose searching 
it on Amazon.com for purchase (Matz, 2014). Similarly, the DOD has an invested 
interest in the technology’s benefit such that the Naval Research Lab (NRL) has created a 
prototype. 
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Computer vision, image proeessing deteetion and reeognition, and maehine 
learning have inereasingly beeome of great interest to the DOD for general intelligenee, 
target reeognition, and data analysis. Currently, the NRL elaims to have developed a 
“rapid reeognition and loeation of surfaee shapes in range images” (Manak, 2014). 
According to the NRL, the technology can differentiate parts in a clutter environment 
with accurate recognition. 

Accordingly, the object/shape recognition technology from Amazon and NRL are 
two examples from the private and government sector. The U.S. government and private 
sectors’ interest in recognition technology is for enhancement in their internal processes’ 
efficiency or to carve a competitive edge. Their interest in the benefits attained by the 
technology ensures that gaps are filled to evolve the object/shape recognition technology. 
Additionally, with the advancement of new commercial technology and resourceful 
databases, there are opportunities to mature object/shape recognition technology. 

H. CHAPTER SUMMARY 

With technology rapidly evolving and the increasing need to sustain a competitive 
edge, the federal government must keep abreast of technological changes. It is beneficial 
to ensure that both the government and the industrial sector maintain open lines of 
communication in order to collaborate and meet efficiency requirements. This chapter 
reviewed the elements that may impact the decision-making with regard to improving the 
current property processing systems. Specifically, this chapter reviewed the generally 
accepted processes for technology commercialization, CBA, U.S. government’s 
assessment of technology transition, technology readiness and, statutory policies, 
directives and regulations. 

The take away is that TC differentiates the technology to target the appropriate 
consumers or users. Parameters or measuring techniques are in place to reduce risk, 
control cost, and creates timeline. The evolutionary process begins from concept, 
continues onto the lab for development, and ends with an operational product for 
warfighters to use in the actual field environment. The TT process is one of the main 
frameworks that effectively incorporates the system or product. TRLs establish 
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parameters to evaluate technology’s maturity. In general, a technology is claimed to be 
mature for production, or roll-out in the case of software, when it accomplishes TRL 7. 
However, DOD considers TRL 6 the benchmark to proceed with system development. 
The statutory codes and DOD guidance provide direct orders to follow with regard to 
considering the commercial industry’s technological advancement and adapting it to the 
government’s needs. All the theories, guidance, and concepts reviewed in this chapter are 
important elements to refer to and which guided this study in answering the research 
questions. The focus of the subsequent chapter is to describe the research approach and 
methodology that lays the foundation of the analyses for conducting this study. 
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III. METHODOLOGY 


A. INTRODUCTION 

This chapter presents in detail the analysis steps taken to address this project’s 
objectives, to conduct an analysis of the DLA DS property process and evaluate the 
technology readiness level (TRL) in order to assess the feasibility to implementing a 
property identification technology, and to conduct a cost-benefit analysis (CBA). This 
applied research study utilizes mixed methods. Mixed methods are used to conduct three 
analyses. First, the study uses a process analysis to understand DLA DS’s property 
disposal operational process. Second, it also uses a technology readiness assessment 
method to assess the viability of implementing object/shape recognition technology by 
the DLA DS. Finally, the study utilizes a CBA method in order to assess DLA DS’ 
property identification operational cost and benefit values provided with technology and 
without technology for the purpose of financial analysis. Each method and its respective 
data sources for each are described in the following sections. The approach’s purpose is 
to set a baseline to conduct the qualitative and quantitative analysis. 

The main objective of an applied research is to explore a practical situation’s 
solution to a problem (Booth, Colomb, & Williams, 2008). The applied research has two 
major components. One component is knowing how to account for factors in identifying 
the problem; the second is doing —rectifying the problem with a solution (Booth et ah, 
2008). This thesis’ focus of effort is to find a solution to the DLA DS property 
identification process determine feasibility in order to apply technology to the process 
therefore; for this reason the researcher designed this applied research methodology. 

B. PROPERTY DISPOSAL PROCESS ANALYSIS METHODOLOGY STEPS 

This section underlines the steps by which the researcher analyzed the property 
identification process. Currently, DLA DS disposes of items using a seven-step process. 
Each step was analyzed to understand the process flow and to identify inefficiencies and 
problems. 
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1 . 


Data 


The researcher visited DLA DS at Camp Pendleton to observe procedures for 
processing property. The researcher visited the site and observed the process for three 
hours and asked clarifying questions to document the process. During the visit the 
researcher took detailed notes and pictures and identified relevant documents such as 
standard operating procedures, training manuals and reports from the Government 
Accountability Office (GAO). These reports were then collected to provide data for the 
study. 


a. Data from Previous Studies 

In 2008, DLA DS captured investigated DS property process. During that study 
one of DLA DS facilities was observed processing property (see Figure 4; DLA DS, 
2008). The result from the 2008 study is particularly important because it provides a 
sample number and the time it took to accomplish each task. Steps of focus for this thesis 
are 5, 6, and 7. Step 5 contains the sample size of items 25 items’ local stock numbers 
(LSNs). Steps 6 and 7 contain the time it took to match and research LSN’s. Although 
plenty of analysis can be generated from the “time study” data, the interest of this thesis 
is to calculate the average time to research each LSN item that needed further 
verification; therefore, the following represents the generic average formula: 
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Figure 4. Process Flow Time Study per Step (from DLA DS, 2008) 


Note: Average researched time to per LSN = Total time to research LSN’s / total number 
of LSN’s researched 

Depicted in Figure 4 is the average time employees take during each step for one 
cycle of property processing. A description of the columns 1 through 5 is as follows: 
responsible employee completing the task, step in the process, time to accomplish task in 
the respective step, description of task process undertaken by employee, and notes 
identified during the study, respectively. The carrying time weight per step in Steps 5, 6 
and 7 is notable. Step 5 takes the longest to accomplish while Step 6 carries the third- 
most time in the cycle. 

The most recent study was conducted in 2013. The study resulted in streamlining 
the property process by up 35% (see Figure 5; DLA DS, 2013). This information is 
particularly relevant to this thesis because it provides the standard by which employees’ 
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performance is measured. The measurement relevant for this thesis is the final average 
resulting in 7.5 receipts per person per hour. These results were utilized to calculate hours 
spent processing items in the item processing analysis. 


Receipt Process Data (Property Entering CDD gate to Property Stowed) 

(1)Day (2) Date 

(3) # Incoming 
Trucks & 
Badged Trucks 

(4) Net # of 
Receiving 
FTE TDY 

(5) Items 
"Received" 

(6) =5] 4 

Mon 19-Aug-2013 

2 

10 

321 

32.1 

Mon 26-Aug-2013 

12 

9 

329 

36.6 

Tue 20-Aug-2013 

11 

9 

138 

15.3 

Tue 27-Aug-2013 

16 

10 

564 

56.4 

Wed 21-Aug-2013 

6 

10 

526 

52.6 

Wed 28-Aug-2013 

13 

11 

636 

57.8 

Thu 22-Aug-2013 

18 

9 

811 

90.1 

Thu 29-Aug-2013 




#DIV/0l 

Fri 23-Aug-2013 

5 

7 

589 

84.1 

Fri 30-Aug-2013 




#DIV/0! 

Totals: 

83 

75 

3914 

52.2 




.'.Average Receipts per Person per Hour: 7.5 


Figure 5. Study to Measure per Person Hours per Property Received for Process 

(from DLA DS, 2013) 


Figure 5 illustrates in a four-week period the average receipts of each person per 
hour, 7.5. Column 1 indicates the day of the week while column 2 is the date; column 3 
illustrates the number of trucks for the day; column 4 records the full-time equivalent 
(FTE) and/or people doing the work who are on temporary duty assignment (TDY); 
column 5 shows how many items were received, and column 6 is a ratio of items received 
divided by FTE received (col 5/col4; DEA DS, 2013). 

b. New Data, Collected from DLA DS at Camp Pendleton 

The author of this thesis travelled to Camp Pendleton, CA, and collected two 
types of data sets that cover four months of property processing information. One data set 
contains data on the total amount of unidentified items that were researched during the 
week. The second data set contains daily, weekly, monthly and annually data on 
identified (input) and processed (output) items at Camp Pendleton staring with date 1 
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July 2014 until date 21 October 2014. The summaries of the two data sets are presented 
in Figures 6 and 7, respectively. The main take away from Figure 6 is the total number of 
researched LSNs that were reported on July 14 used to calculate a historical quantitative 
relation between items researched and items received. This calculation of the ratio 
number is presented in Figure 7. The generic ratio formula is as follows: 


Researched LSN List 


PARENT DTID: 


Aooroved As 

COMMENTS 

CHILD DTID NUMBER: 

LSN 

Demil Code 

NSN Found 

R5709440360046 

811500CASE 

A 

6760-01-519-0711 

N6312640525C5J 

N6312640525C5J 

D 


N6312640S25N5J 

5840DSRADAREQ 

D 


N6312640101FSJ 

N6312640101F5J 

A 


M93338-4195-D303T 

61.30-DSUPS00033 

O 


£2128041272416 

6636DSENVCHMBR 

A 


N6312640525H5J 

6605DSNAVIGAT 

0 


N6312640525GSJ 

5840DSRADAREQ 

D 


N6312640303B5J 

5B40DSRADARFQ 

D 


W900DB41710001 

518001C047509 

A 


FX231533390021 

1560OSPANEL00 

D 


M3380042020018 

SieODSTOOLKIT 

A 


Di»D. Svcs Site: 

7/14/14 



The LSN('s) listed on this form have been Researched and Demil Code Verified as 
indicated above. The LSN(s) above have been authorized for release lAW DLA 
Dispo Svcs Policy. 





Printed CPB Certifters Name 



CPB Certifier Signature 













: Printed CPB Verifiers Name CPB Venfier Sionature 













Note: Please make sure you are reviewing Special Processing Requirements for item(s) that may require Special Processing 
It is each Field Sites responsibilKy to ensure special processing requirements are met 

Items that are not able to be Verified by CPB shall be downgraded according to Policy. 


Figure 6. Researched Local Stock Number, 7/14/14 (from DLA DS, 2014) 


Note: Ratio of LSN items researched in a week compared to total items received = Total 
LSN’s researched (sum LSN in Figure 6) / total number of receipts for the week (weekly 
sum of receipts from Figure 7) 


31 





























A 

B 

- 

D 

E 

F 

c 

H 

1 J 









1 

Jul 

through... 

7/31/14 

sorted by Region, Site & Dete 

Point In 

Time 

RCN Open 

Point In 

Time 

RCN AGE 

RCN to 

Induct 

(RCNS) 

Point in Time 

EtacUog Ending 
(Lines) 

Point in Time 

Receipts 

(Lines * Inventory 

Batches) Levd (Lines] 

ADD N«w DAILY 

D*ts to bottom 
(PoLte Values), 
then Sort by 

R«(ion, Site, Dece 






Date 

Region 

Site ID 

Site 

Count 

Oldest 

RCN 

DaysAvg 

Sadciog Count 

Receipts Inventory 

7/2/14 

West 

RCCP 

DLA DS PENDlfTON 

30 

12 

8.6 

1,104 

256| 3.818 

7/3/1* West 

RCCP 

DU OS PENDLETON 

30 

12 

8.6 

- 


7/4/1* 

West 

RCCP 

DLA DS PENDLETON 

30 

12 

- 

- 

b T 

7/5/1* West 

RCCP 

DIAOS PENDLETON 

30 

u 

- 


’9 i=J 

7/€/14 

West 

RCCP 

OLA DS PENDLETON 

28' 

12 

- 

i.703 

b 1 3.824] 

7/7/14 West 

RCCP 

DLA DS PENDLETON 

3i 

13l| 33 

2,320 

23ll| 3,812| 

7/8/14 West 

RCCP 

DLA DS PENDLETON 

31 

14 

10.0 

2.417 

3571 3,487 

7/9/14 West 

RCCP IDLADS PENDLETON 

34 


7.9 

2,507 

3111 3,508 

7/10/14 

West 

RCCP 

DLADS PENDLETON 

37 

14 

3.6 

2,530 

298 1 3,539 

7/11/14 West 

RCCP 

DLA DS PENDLETON 

37 


8.S 

- 

2W|- 

7/12/14 
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RCCP 

OLA DS PENDLETON 

SI 

14 

- 

~ 

B 

7A3/14 West 

RCCP 

OLA DS PENDLETON 

40 

1 17 

“ 

2,538 

15 ~r 

7/14/14 

West 

RCCP 

DLA DS PENDLETON 

44 

15 

14 3 

2.353 

3511 ^3.39^ 

7/15/14 West 

RCCP 

DLADS PENDLETON 

47 

16 

12.5 

2,1m 

333) 3/477, 

7/16/14 

West 

RCCP 

DLA DS PENDLETON 

51 

17 

13,5 

2.055 

30311 3,452 

7/17/14 West 

RCCP 

DLA DS PENDLETON 

SI 

18 

125 

2,099 

3011 3,014 

7/18/14 

West 

RCCP 

DLA DS PENDLETON 

51 

18 
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- 
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h 

^ 18 
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- 
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45 

18 

- 
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19 
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1S7) 3,096] 
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54 

16 
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2,431 
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7/23/14 West 

RCCP 
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17 

6.4 
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RCCP 
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51 

18 

4.9 
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51 

18 

6.4 

- 

4011- 
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West 
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DLA DS PENDLETON 

-- n 


n 1- 1 
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0.0075 
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Figure 7. DLA DS at Camp Pendleton Daily Counts of Inventory Processed during 

July 2014 (from DLA DS, 2014) 


Data presented in Figure 7 was analyzed to determine the effects of the Receipt 
Control Number (RCN) on input. Open RCN, RCN age, RCN to induct, ending backlog, 
running receipts and ending inventory are illustrated in columns E-J, respectively. Open 
RCN accounts for RCNs inducted to day and are being tracked. Oldest RCN indicated 
days oldest RCN aged; items are processed using the first-in, first-out method. RCN to 
induct indicates the average amount of days to process an RCN; per guidance it should be 
processed in three days. Backlog ending, reports an estimate amount of items that are 
stowed for future process. Receipts, reports total amount of items processed for the day. 
Inventory level refers to total amount of items in the warehouse that have been processed 
and are usable. The interest of this thesis is to analyze item input and objectively assess 
the output. Moreover, for this thesis, the amounts of employees are considered with 
regard to average per employee per hour that can processing items for disposal, thus the 
number of employees that are employed to process items impact the output. The 
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remaining items will be considered to be backlog for future processing. The coefficients 
are explained by the simple formula: 

backlog = input - output 

To illustrate backlog trend a broad understanding of the sales backlog formula 
was adapted and modified to illustrate items backlog rather than sales backlog (Bragg, 
2012). The researcher sought to identify items’ backlog worth of work days for a crew of 
six employees, therefore the sales formula was modified to fit DLA DS property backlog. 
The result is related to the rate of the estimated number of items received then stowed, 
and the rate of items processed. So, the backlog is the number of items that remain 
unprocessed in a time span. The formula is as follows: 

_ , , ^, Backlog items 

Days worth of backlog =-^- 

Processed items per day 

The ratio considers the number of items backlog divided by the number of items 
processed (receipts) per day. If the number of backlogged items is greater than the 
number of items processed (receipts), then the ratio will be greater than 1 day. This 
indicates that the facility receives more items and cannot keep up with more than the 
demand than it processed. Conversely, if the number of backlog items is less than the 
number of items processed (receipts), the ratio will be less than 1 day. This indicates that 
the facility can keep up with demand to process as many items as it receives. 

Both data sets provided by DLA DS have relevance to this study because they 
contain metric measurements captured on a daily operational basis. Therefore, the data 
analysis provides meaningful empirical evidence to compare during the quantitative and 
financial analysis for the CBA. 

c. Data Normalization Procedure 

One of the most challenging tasks when collecting primary raw data is the data 
normalization process, because one needs to be careful not to adjust the data and change 
its empirical value. In order to attain better approximation of time-task-duration the 
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researcher drew from the Program Evaluation and Review Technique (PERT) and the 
Pearson Tukey formula to conduct data normalization. PERT estimates time in an 
uncertainty scenario and to best define the time mean and standard deviation for task 
duration the Pearson Tukey technique is appropriate (Klastorin, 2011). Both the standard 
deviation and the mean are assumed to be normally distributed. 

Given than some the data collected was from a subject matter expert (SME), 
Klastorin (2011) suggested that experts have difficulty estimating with great accuracy. 
The SME at DEA DS in Camp Pendleton is the only trained verifier who has the 
credential to access databases to conduct the item research. Therefore, the verifier is 
within bounds for the author to rely on his expert opinion. However, Klastorin (2011) 
further suggested utilizing the Pearson Tukey technique to reduce subjectivity in the 
SME’s opinion. The Pearson Tukey formula considers error in the estimation’s mean and 
variance calculations. The resulting formula consider these factors, as shown below: 

Mean Task Time = tso + 0.185(t95 + ts -Itso) 

Standard deviation = - t5)l3.25 

where: tso = median 

t 95 = 95th Percentile 
= 5 th Percentile 

Traditionally, the Monte Carlo simulation approach computes the probability 
safety lead-time that the task will be completed on time (Klastorin, 2011). Therefore, in 
the search to fully estimate a task’s time completion, this study used the Monte Carlo 
technique. Eurthermore, the researcher used the Crystal Ball add-on application to 
Microsoft Excel to simulate the scenario for 10,000 times. Then, a lognormal distribution 
parameter was used in order to get positive time result as only non-negative values are 
meaningful in this case. The distribution is skewed because it doesn’t give negative date 
and time that are not possible occurrences (GAO, 2009b). Eigure 8 presents the 
distribution parameter’s description and application justification. 


34 



Distribution 

Description Shape 


Typical application 

Lognormal 

A continuous distribution 
positively skewed with a 
limitless upper bound and 
known lower bound; skewed to 
the right to reflect the tendency 
toward higher cost 

Probability 


To characterize uncertainty 
in nonlinear cost estimating 
relationships; it is important to 
know how to scale the standard 
deviation, which is needed for this 
distribution 



Values 

Normal 

Used for outcomes likely to 
occur on either side of the 
average value; symmetric 
and continuous, allowing for 
negative costs and durations. 

In a normal distribution, about 

68% of the values fall within one 
standard deviation of the mean 

Probability 

// 

, / 

To assess uncertainty with cost 
estimating methods; standard 
deviation or standard error of the 
estimate is used to determine 
dispersion. Since data must be 
symmetrical, it is not as useful 
for defining risk, which is usually 
asymmetrical, but can be useful for 
scaling estimating error 



Values 


Figure 8. Probability Distribution Description Shapes and Application 

(from GAO, 2009b) 


Finally, adjusting the data for uniformity to actual cost is considered. Given that 
some of the data was generated in 2008 and other data in 2013, normalization is required. 
Similarly, the historical dollar value data must be normalized for inflation. Therefore, the 
researcher used the military construction (MILCON) inflation factor to convert the 
funding profiles and determined the then year (TY) into fiscal year 2014 (FY14) to 
establish a baseline of cost. Additionally, the default setting “Show empty data as zero” 
was enabled in Excel to automatically treat blanks cells as zero. Appendix B showcases 
the treatment of empty cell the method. Furthermore, the results from the data 
normalization were used to estimate the time-task-duration to account for it in the CBA 
section of the thesis. 

The following disclaimer should be noted: Crystal Ball is licensed for educational 
purpose by The Naval Postgraduate School; therefore, the Crystal Ball charts contain the 
statement “Not for Commercial Use.” 

In summary, normalizing reduces dispersion of data points and maintains constant 
and accurate information. Therefore, it is imperative to adapt the method and techniques 
described in this section in order to be applied in this study. 
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c. 


TECHNOLOGY READINESS METHODOLOGY STEPS 


Using the TRL table established in Chapter II, this study provides results of the 
three companies. According to the GAO (2003), the TRL tool is used to assess 
technology’s maturity. The TRL methodology is widely known in the DOD for its best 
practice approach of ranking technology’s maturity. The benefit of evaluating three 
companies’ technology provides controls to reduce the risk of running the project over 
schedule and over budget. The foundation to make the technology assessment using the 
TRL tool was outlined in Chapter II. 

In this study three commercial-off-the-shelf object/shape recognition tools were 
evaluated. This study followed the TRA skeletal template from the DOD TRL Guidance 
and adopted the DOE report format. The COTS with the highest TRL will be 
recommended for further maturation. The TRA report consists of the following 
information: 

1. Overview of object/shape recognition technology profile. 

2. Technology’s goal with regard to what it does and affects. 

3. Benefits with regard to how the technology benefits businesses that use it. 

4. Critical area(s) of challenge such as finding a demonstration and any risk 
implementing it at DLA DS. 

5. Qualitative summary of technology’s technology readiness level (TRL) 
using the TRL table in Chapter I. 

6. Statement of relevancy to DLA DS, which provides a conclusion with 
supporting evidence or rationale and dissenting opinion of technical 
aspects information that may be deemed pertinent to support the decision¬ 
maker. 

1. Data Sources for Technology Readiness Level Assessment 

The researcher visited three companies during the months of August to collect 

data to assess the technology maturity of three products. The researcher spent one half 

day at each of the company. The researcher observed demonstrations of the technology 

products at each firm, taking detailed notes and asking clarifying questions to develop a 

deep understanding of each product’s capabilities. The researcher also collected 
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presentations of the technologies, and white papers, pamphlets and videos describing the 
technologies. This data is summarized in Table 2. 


Table 2. Data Collected from Companies 


Company 

Data 

Quantity 

COGNEX 

Pamphlet 

2 

COGNEX 

Video from YouTube 

5 minutes 

Imaginestics 

Pamphlet 

2 

Imaginestics 

Pictures 

10 

Imaginestics 

PowerPoint (1) 

10 slides 

Imaginestics 

Video 

20 minutes 

RMD 

White paper 

1 


Overcoming Data Challenges: The researcher gathered the primary data and 
information used for this study during business trips. The purpose of the business trips 
was to observe and assess the technology’s maturity, and conduct a cost evaluation. The 
researcher visited three potential companies of interest and communicated via telephone 
and email with three other companies. Admittedly, there were challenges in gathering 
actual technology costs posed by the companies’ leadership due to proprietary technology 
rights, privacy policies and reluctance to advertise actual cost without negotiating a 
contract. However, one company did provide examples of costs for use in this thesis. 
Finally, the researcher believes that the evidence obtained during the study provided 
reasonable basis to provided reasonable accurate information in order to make an 
informed analysis conclude with recommendations. The study does not endorse one 
specific company over another. The intent of the study is to offer DLA DS decision¬ 
makers as much information relating to what the commercial sector has to offer on 
object/shape recognition technology as possible, so that they can make the most informed 
decision about implementing technology to their property identification process. 

D. COST BENEFIT ANALYSIS METHODOLOGY STEPS 

The CBA considers which alternative provides the highest positive tangible and 
intangible realized net benefits. The CBA is an analytical tool to support the decision- 
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making process when choosing between competing alternatives. A sensitivity analysis 
supports the CBA by changing assumptions to understand risk factors that alternatives 
inherit. Boardman et al. (2011) proposed the analysis to calculate social benefit for each 
alternative: net social benefit (NSB) equals social benefit (B) minus social cost (C). 


NSB = B - C 


According to Boardman et al. (2011), there are three major types of CBA and one 
comparative class of CBA: 

1. Ex ante CBA: conducted prior to the project; useful to show whether 
resources should be allocated for a particular project. 

2. Ex post CBA: conducted at the end of the project; provides information 
about the particular class of project. 

3. In medias res CBA: conducted during the project. 

4. Comparative CBA: compares the ex ante predictions to ex post results for 
the same project. 

This research study used the ex ante CBA to provide information to DEA 
decision-makers prior to making an investment. Notwithstanding, there are nine basic 
steps in conducting a CBA (Boardman et al., 2011, p. 6), which will be applied in the 
methodology chapter. The details of all the steps for CBA are detailed in the sections that 
follow: 

Step 1: Specify the set of alternative projects. 

Step 2: Decide whose benefits and costs count (standing). 

Step 3: Catalog the impacts and select measurement indicators. 

Step 4: Predict the impacts quantitatively over the life of the project. 

Step 5: Monetize (attach dollar values to) all impacts. 

Step 6: Discount benefits and costs to obtain present values. 

This analysis normali z ed the wages data for inflation. Therefore, a nominal 
discount rate is used to discount cost and benefit over the 10-year horizon time. 
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Furthermore, in order to account for the time value money, this research used a 7% 
discount rate. According to the Circular A-94, when analyzing proposed investment the 
7% rate is ideal for calculating the return on investment in the private sector (0MB, 
1992). 

Step 7: Compute the net present value of each alternative. The net present value 
(NPV) equals the present value of benefits minus the present value of costs: 

NPV = PV (B) - PV(C) 

Choose the alternative with the largest NPV. 

Step 8: Perform sensitivity analysis. To shield from uncertainty of the predicted 
impact, this study conducted a yearly variance sensitivity analysis using the 5% lower 
bound and 90% upper bound confidence intervals. 

Step 9: Make a recommendation. The results from the NPV calculation with the 
alternative with the largest NPV will be the criterion for recommendation. This will also 
consider the sensitivity analysis to make recommendations. 

For this thesis, the status quo alternative is compared with the option to 
implement new object/shape recognition technology. This thesis builds upon this 
generally accepted approach to fully analyze the costs and benefits associated with 
implementing object/shape recognition technology at DLA. As a general rule, CBA is 
created for a comprehensive analysis. 

1. Assumptions, and Data Sources for Cost-Benefit Analysis 

The following data and assumptions are used for conducting the cost benefit 
estimates. 


a. Assumptions 

In order to estimate benefits generated by the implementation of an object/shape 
recognition technology, this thesis uses the opportunity cost approach. In essence, the 
benefits associated with the new technology are estimated as savings in labor costs that 
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do not have to be paid out any longer once the labor-intensive status quo is replaced. 
Although DLA DS is a non-profit organization, the researcher is able to use the market 
comparable approach to calculate the return on investment. While DLA DS does not have 
an overall comparable commercial market force to compare revenues, the above 
assumption allows the researcher to use the market comparable labor rates as a form of 
estimating revenues (Housel et ah, 2007). In order to conduct a break even or return on 
investment analysis, Jegers (2008) suggested calculating earned revenues from other 
sources rather than sales. Therefore, there seems to be no reason to use the current 
general schedule (GS) wages as market comparable revenues. The researcher calculated 
the annual wages in dollars per hours in order to monetize the LSN research task, the 
assumption being that automating the identification process with technology will save 
money in labor cost and increase productivity. Additionally, implementing object/shape 
recognition technology will not replace the bar code readers because implementing 
object/shape technology is for the identifying items without information. Furthermore, 
the scope of this thesis limits the amount of variables one can research and utilize in 
accounting for how the technology will generate more market comparable revenues. 

b. Data on Employee Wages and Number of Disposal Operations 
Employees 

The researcher drew from the GS pay chart to indicate the average labor rates for 
employees who conduct disposal services at DLA DS, Camp Pendleton, California. The 
cost of labor is used to conduct the NPV to satisfy a market comparable approach in 
terms to calculate the return on investment. The GS pay table is illustrated in Figure 9. 
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SCHEDULE 1—GENERAL SCHEDULE 


(Effective on the first day of the first applicable pay period beginning on or after January 1, 2014) 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

GS-1 

$17,981 

$18,582 

$19,180 

$19,775 

$20,373 

$20,724 

$21,315 

$21,911 

$21,934 

$22,494 

GS-2 

20,217 

20,698 

21,367 

21,934 

22,179 

22,831 

23,483 

24,135 

24,787 

25,439 

GS-3 

22,058 

22,793 

23,528 

24,263 

24,998 

25,733 

26,468 

27,203 

27,938 

28,673 

GS-4 

24,763 

25,588 

26,413 

27,238 

28,063 

28,888 

29,713 

30,538 

31,363 

32,188 

GS-5 

27,705 

28,629 

29,553 

30,477 

31,401 

32,325 

33,249 

34,173 

35,097 

36,021 

GS-6 

30,883 

31,912 

32,941 

33,970 

34,999 

36,028 

37,057 

38,086 

39,115 

40,144 

GS-7 

34,319 

35,463 

36,607 

37,751 

38,895 

40,039 

41,183 

42,327 

43,471 

44,615 

GS”8 

38,007 

39,274 

40,541 

41,808 

43,075 

44,342 

45,609 

46,876 

48,143 

49,410 

GS-9 

41,979 

43,378 

44,777 

46,176 

47,575 

48,974 

50,373 

51,772 

53,171 

54,570 

GS-10 

46,229 

47,770 

49,311 

50,852 

52,393 

53,934 

55,475 

57,016 

58,557 

60,098 

GS-11 

50,790 

52,483 

54,176 

55,869 

57,562 

59,255 

60,948 

62,641 

64,334 

66,027 

GS-12 

60,877 

62,906 

64,935 

66,964 

68,993 

71,022 

73,051 

75,080 

77,109 

79,138 

GS-13 

72,391 

74,804 

77,217 

79,630 

82,043 

84,456 

86,869 

89,282 

91,695 

94,108 

GS-14 

85,544 

88,395 

91,246 

94,097 

96,948 

99,799 

102,650 

105,501 

108,352 

111,203 

GS-15 

100,624 

103,978 

107,332 

110,686 

114,040 

117,394 

120,748 

124,102 

127,456 

130,810 


Figure 9. General Sehedule Pay Table for 2014 
(from Exeeutive Order No. 13,655, 2013) 


The DLA DS at Camp Pendleton structure was collected to identify how many 
employees are part of the disposal operations. During the study, two positions were 
vacant as illustrated in Table 3. The two employee functions include the Disposition 
Services Representative (DSR) supervisor or a DSR, the pay plan is under the GS rules in 
accordance to the series standard 1104 classifying the employee as a Property Disposal 
Clerical or Technician pay grade hired under the available positions.) In this case, the 
grades are GS 9 through GS12. Figure 9 identifies wages for every respective position. 
For this study, the GS 11 is step 6, whose salary is $59,255, consistent with the pay scale 
the verifier at Camp Pendleton holds. 


Table 3. DFA DS at Camp Pendleton, California, Disposal Operations 
Structure as of July 2014 (from DFA DS, 2014) 


Disposa 

Operations Si 

tructure 

Vacant/Full 

Function 

Pay Plan 

Series 

Grade 

Vacant 

DSR 

Supervisor 

GS 

1104 

12 

Full 

DSR 

GS 

1104 

11 

Full 

DSR 

GS 

1104 

11 

Full 

DSR 

GS 

1104 

09 

Vacant 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

09 

Full 

DSR 

GS 

1104 

11 
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E. SUMMARY 


This chapter outlined the foundational method for conducting the CBA in order to 
make informed recommendations. This ehapter deseribed the steps taken to analyze the 
item proeessing; including the method used to normalize the data to ensure uniformity 
and integrity. The ehapter also presented the assumptions and data sourees for the eost 
benefit analysis. The researeher took into aeeount that eolleeting data from a SME has 
biases sueh that to reduee their opinion’s subjeetivity the Person Tukey is an appropriate 
formula to consider for error. The next ehapter will present the analysis of DLA DS’s 
purpose and operational method, and diseuss the assessment results on COTS of 
objeets/shapes recognition technology. 
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IV. ANALYSIS OF THE CURRENT PROPERTY PROCESS AT 
DEEENSE LOGISTICS AGENCY DISPOSITION SERVICES 
AND MATURITY ASSESSMENT OE OBJECT/SHAPE 
RECOGNITION TECHNOLOGY 


In support of the DLA mission and vision, DLA Disposition Services 
supports the Warfighter and protects the public by providing worldwide 
reverse logistics solutions. Our goal is to become the Department of 
Defense’s Reverse Logistics Center of Excellence 

—DLA Disposition Services Transformation Plan 2014-2019 

A. OVERVIEW 

Chapter III explained the method by which this thesis explored the organization 
process with Defense Logistics Agency Disposition Services (DLA DS). This chapter 
presents the background on DLA DS management’s concerns to contextualize this study, 
and presents and discusses the results of that analysis of the current property process at 
DLA DS. In addition, this chapter provides an overview of DLA DS’ purpose, 
operational process activities, and managements’ concerns and details the results from the 
assessment on objects shapes recognition technology prototypes from three companies. 

1. Introduction 

On September 1972, the Disposition Services (DS) was established as a division 
of the Defense Logistics Agency (Grasso, 2014). The mission of the DS is the 
management of the DOD’s assets that have reached the end of their life cycle. 
Specifically, DS focuses on solutions to dispose property no longer wanted from all the 
Marines, Army, Navy, and Air Lorce, and other federal agencies. Today, the DLA DS 
management is interested in improving the efficiency of the item processes. One potential 
way to improve efficiency of the DLA DS processes is the use of object/shape 
recognition technology, identifying as a way to reduced costs by DLA DS (2008) study. 
This section describes the DS purpose and motivation for considering technology to 
achieve DLA DS goals and underlines the property identification process, inefficiency, 
problems with the processing system that creates backlog, and analysis to resolve the 
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backlog issue. To illustrate the points, observations made by the researcher during the 
visit at the DLA Disposition Services at Camp Pendleton, California, will be provided. 

2. Defense Logistics Agency Disposition Services Activity 

a. Disposition Services Purpose 

The DS provides support to the DOD in 109 different locations worldwide that 
including 15 countries and 41 states, employing over 1,500 employees. DS Headquarters 
is in Battle Creek, Michigan, its location meant to control and support its mission (DLA 
DS, 2013). 

The broad purpose of a DS network is to facilitate the DOD in property disposal 
around the world. DS calls the services “reverse logistics.” Reverse logistics is defined as 
the operations that accept property from the branches of services to screen it for 
redistribution (DLA DS, 2013). To manage accountability and control around the world, 
DS’s purpose statement says the following: 

Our purpose is to serve as the Reverse Logistics provider for the 
Department of Defense to include services for the disposal of material no 
longer needed for national defense, comply with legislative and regulatory 
requirements, protect the public from dangerous defense items, and pursue 
maximum value for tax dollars. Our Reverse Logistics capacity includes 
responsibility for property reuse (including resale), hazardous property 
disposal, demilitarization, precious metals recovery and recycling program 
support, worldwide. Reverse Logistics is more than property disposal; it is 
the process of planning, implementing, and controlling the efficient, cost 
effective flow of excess materials, in-process inventory and related 
information from the point of declaration of excess for the purpose of 
recapturing value or proper disposal. (DLA DS, 2014a, p. 1) 

Reverse logistics includes the following disposal choices: reuse; transfer property 
within inter branch of service; donate property to state, local or non-profit organization 
and sell it to the public; or destroy it to turn it into scrap to prevent further use. 
Consequently, DS goals are aligned with the DLA to accomplish the reverse logistics 
purpose. 
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b. Motivation for Considering Technology 

As important as DS is to the DLA, the DLA director requires that all divisions 
“Improve process and productivity” in order to become responsible stewards of 
taxpayer’s dollar (DLA, 2014). Additionally, DS has the added pressure of fiscal 
obligations and budgetary cuts such that DS continuously pursues decreasing operations 
cost. Therefore, goal achievement and sustainment is of DS top priority, so factors that 
affect operation cost are continuously examined. 

To accomplish reductions of operations cost, DS conducts studies to identify any 
cost-reducing opportunities. In 2008 (DLA DS, 2008), a study to change the 20-step 
property acceptance process was concluded at one of DS’s facilities located in Stockton, 
California. The study resulted in streamlining the process to seven steps using the Lean 
Six Sigma logic. The change in process resulted in time savings equivalent to 40% (DLA 
DS, 2008). The study concluded with a recommendation for a future study to exploit 
wireless capabilities (DLA DS, 2008). 

This thesis aims to analyze DS item process, conduct a technology readiness 
assessment and conduct a cost benefit analysis in effort to finding a solution to improve 
the process with technology. To achieve this goal, this thesis adopted two DS strategic 
objectives set by the DLA DS Director Transformational Plan 2014—2019: 

1. Embrace innovation and leverage industry: “Employ industry and 
academia to pursue technological advancement in reverse logistics 
solutions” (DEA DS, 2014a, p. 10). 

2. Identify opportunities for process enhancement to reduce cost: “Optimize 
Network to provide accountable, compliant, and agile property handling 
(internal operating system strategy)” (DEA DS, 2014a, p. 10). 

3. Property Disposal Process 

In 2008, efforts to improve the property process resulted in a streamlined process 
that saved time through the lean approach. Currently, DS continues to process property 
using the Eean Six Sigma standards. However, DS recognizes that the improved seven- 
step process can be tweaked; therefore, DS is interested in additional improvements. 
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Hence, in order to seek improvements, one needs to understand the process flow to 
identify inefficiencies, drawbacks, and problems. 

DS manages property for disposal and the choices include one of the following: 
reutilization, transfer, donation, resale and ultimate disposal as scrap. In this context, the 
holistic process is depicted in Figure 10. 


Army 
Navy^ 
Marine Corps -» 
Air Force 
DLAyOther 


Receive 

Physical 

Turn-in 


Electronic 

Turn-in 


Hold 


Screening 
Item Verification 
Demilitarization 
Downgrades 



r* Reutilization 


*1 Transfer 


. Army 
Navy 

. Marine Corps 
■ Air Force 
' DLA/Other 
' Federal Agenciesi 


Donation 


Usable Sales 
Scrap Sales 


State Agencies/ 
Organizations 

Public 
■ Public 


Ultimate Disposal (UD) 


Figure 10. Disposition Services Process Disposal Overview (from DLA DS, n.d.) 


Figure 10, presents the overall flow of property acceptance conducted at DS. The 
items are received from all four branches of service—Army, Navy, Marine Corps and Air 
Force—and other federal agencies. Then the items are individually screened for proper 
disposal decision. Some of these items are redistributed for reutilization by any branch of 
service or transferred to federal agencies. Major recipients of donated items go to state 
and local agencies, like the fire department or police stations. Additionally, some items 
that are safe for public use are sold through public auctions and scraped items are sold 
through business partners. Finally, items that cannot be sold to the public due to potential 
hazard or to prevent putting them into the hands of an adversary are mutilated or stored in 
military facilities. 

The property disposal process begins when the customer completes the mandatory 
disposal turn-in document (DTID), DD Form 1348 lA, and associated certificates 
depending on the type of property (further details about how the document is generated is 
emphasized later in the section). The property is accepted from the customer as a 
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conveyance and tagged with a receipt control number (RCN). The RCN identifies the 
customer, tracks the time it takes to process the items, reduces dual effort, organizes for 
precedence processes, order, and optimizes limited resources. Conveyance is then tracked 
and identified as an RCN. 

Turned-in property at DS arrive in pallets, triwalls full of line items (Lis) or single 
LI. DS employees are hired in any position under the Deposition Service Representative 
(DSR) and are known and referred to full-time employees (FTEs). FTEs store the RCN 
property that cannot be inspected for disposal decision upon arrival, and at that moment it 
becomes backlog for future process in precedence of arrival. Items that are inspected on a 
daily basis are broken down into individual Els, and then the usable property is retained 
for 42 days of screening. During the screening phase, the retained property is marketed 
for reutilization, transfer, or donation (RTD) to the branches of services, or local or 
federal agencies, as noted in Figure 1. After 42 days, the El that is not RTD is handled 
again and turned over to government liquidations contractors to sale to the public. 
Property that requires demilitarization (DEMIE) is only available for a 14-day period per 
the DOD screening cycle and then sent to a DEMIE site for proper disposal and 
destruction. DEMIE property is not made available for public sale. Figure 14 illustrates a 
timeline of the process. 



Figure 11. Disposition Services Process Disposal Timeline (from DEA DS, n.d.) 


Figure 11 displays the days required to accomplish property disposal. The RTD 
activities occur between days 1 and 42. In the incubation period, days 1-7, El is pending 
screening to decide appropriate disposal. Annually, on average, RCNs wait 7.6 days to be 
processed for screened for demilitarization or downgraded for scrap. During the 
reutilization-screening period, 1-14 days, Els are screened for the RTD program. During 
the transfer-screening period, the El is initially requested to be RTD. During the 
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donation period, 36-40 days, Lis are released under the RTD program for reuse. These 
items can then be transferred to the federal agency that needs it. The law authorizes these 
transfers to federal agencies, state agencies, or police stations. Additionally, items can be 
donated to non-govemmental agencies or humanitarian aid organizations. Annually DS 
takes, on average 32 days to ship out under a document identifier form Materiel Release 
Order (MRO). Between 40 and 42 days, last chance, are the last days items can be 
requested for RTD by any branch of service, state agency, or local agency. Finally, 
during ultimate disposal (UD), the end of the cycle, some property can be sold to the 
public when it is deemed safe for sale, destroyed, or transferred to a long-term storage 
facility. The flow diagram (see Figure 12) provides amplifying explanation that occurs at 
every step of the disposal process. 
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Figure 12. Process Flow Diagram of Items Turned in for Proper Disposal 

(from DLA DS, 2008) 


All items turned in for disposal at DS go through these internal sequence of 
events: (1) receive items received into conveyance and issued RCN to store in backlog 
storage, (2) conveyance broken down, (3) screening, (4) item sorting, (5) unidentified 
item research is conducted, (6) code for disposition identified, (7) boxed in tri-wall 
containers for final life cycle procedure. The process depicted in Figure 12 provides the 
steps the Lis undergo. All items turned in undergo through this cycle: 

1. Receive items: Conveyance is tagged, but it may have up to 50 
individual line items. Approximately 70% of all Lis received are 
downgraded upon receipt for sale as scrap by weight (DLA DS, 

2014a). 
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2. Breakdown the conveyance: Batches are itemized into individual Lis 
for inspection. This accounts for the remaining 30% of Lis. These Lis 
end up in the RTD pipeline. 

3. Screen: As noted above, the RTD screening process includes a “42- 
day staggered availability period that offers property for reuse first to 
DOD consumers; next other federal government consumers and last, 
the property is available for donation to authorized nonprofit 
organizations” (DLA DS, 2014a). After 42 days, the items are sent to 
the Control Property Center (CPC) for sorting. Figure 11 presents the 
screening timeline. 

4. Sorting: Batches are sorted, identified for restrictions to sell it to the 
public use demilitarized (scrap) or long term storage. 

5. Managed unidentified item: The FTE verifies and conducts research 
using one or all three databases (i.e.. Federal Logistics Data and FLIS 
= Federal Logistics [FEDLOG], WebEEIS and Haystack). 

6. Code for disposition: The items that cannot be verified are 
downgraded to a scrap category. 

7. Managed boxed-in-tri-wall containers: Items with codes: de mi litarized 
or mutilate (MUT) for total destruction to the point of scrap are boxed 
and sent to the nearest Centralized DEMIE Division (CDD). 

After 42 days, in the last three steps of the process, the ETEs screen items for 
ultimate disposal. Steps 5 and 6 illustrated in Eigure 3 are the focus of this project. Step 5 
is a crucial step because time, resources and money are consumed because the ETE 
conducts research. In Step 6, the ETE decides to scrap unidentifiable Els. Conversely, 
when El information is discovered, the ETE determines to give the El one more 
opportunity to RTD. 

In all, the items are disposed through RTD, resale, scrap, or long-term storage. 
There is no shortage of available Els as items continuously arrive. Similarly, at the other 
end of the process items are disposed but sometimes with challenges. Where does the 
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disposed property end up? According to DLA DS (2014a) these items have a final 
destination and the percentages are explained: 

Of the total items that enter the disposition process, approximately 85% of 
items are disposed as scrap or Hazardous Waste (HAZMAT), 6% of items 
are claimed during the RTD screening cycle and 9% of the items are 
eventually sent to sales when not requested for reutilization. This type of 
property includes Hazardous Material (HAZMAT), Hazardous Waste and 
items with specific DEMIL requirements, all of which are handled under 
specialized rules. A fraction of the items that are identified as DEMIL 
coded “B” or “Q,” (due it being hazardous to release to public), are sent to 
Long Term Storage. (DLA DS, 2014a, p. 7) 

In summary, in 2008, the efforts to improve received property resulted in a 
streamlined process that saved time. Currently, DS continues to process property using 
the Lean Six Sigma standards. DS recognizes the improved 7-step process can be 
tweaked; therefore DS is interested in additional improvements. The next section 
provides observations that aim to identify any inefficiencies, challenges, drawbacks and 
problems with the property process. 


a. Inefficiencies 

Lrom a qualitative perspective, the current process requires multiple 
“touches” of the property including handling, segregating, processing, and 
shipping. This relatively “high touch” and risk adverse environment is 
inefficient, costly and burdensome to our customers and ourselves and 
must be reduced. 


—DLA DS, 2014a, p. 7 

The previous sections described the disposition process and the goals to cost 
savings associated with improving the item disposition process. However, some of these 
efforts have been challenged with the ongoing inefficiency to effectively identify Lis. 
Examining the operational flow in Ligure 3 reveals where potential challenges and 
“bottlenecks” are created. The results of the analysis, it turned out, show that researching 
items is not the only area of concern with regard to process inefficiency. About less than 
1% of accepted items at DS facility in Camp Pendleton are unlabeled which creates for 
an additional time to research the missing information. Actions to dispose of Lis may be 

slow due to the inaccuracy of information recorded in the DTID, accepted LI is obsolete 
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and risk adverse culture creates drawbacks and impacts progress. Additionally, a greater 
number of accepted items may be more than the amount of items that are processed on a 
daily basis. This inefficiency is precisely quoted above and cited from the DLA DS’ 
Transformation Plan 2014-2019 (DLA DS, 2014a). 

In general, items that are properly marked occur when the disposal request, DTID, 
is generated via the Dispositions Services’ website or the Distribution Standard System 
(DSS), which comes from Federal Logistics (FLIS). DSS is the automated property 
accounting and inventory management system used by DS to process personal property 
through the disposal cycle (DLA DS, 2013c, p. 1). When inputting a DTID with a valid 
NSN into DSS, the system automatically populates the corresponding DEMIL code that 
has been assigned in FLIS. If the screen shows a DEMIL code that is different from what 
is believe the code should be, then the customer can make corrections to enter the proper 
information and that alert allows for verification to correct for coding errors so that the 
proper item is submitted for disposal. According to the DLA DS training manual, there 
are benefits for filling the DD 1348 through the web-based Electronic Disposal Turn-in 
Document (ETID) program: 

ETID allows the customer to electronically submit turn-in documentation 
to the disposition services site. This program will pre-populate many of 
the fields for NSN items, nomenclature, DEMIE code, unit price, etc. It 
includes drop down menus for other fields for quick reference. In addition 
to electronically preparing your turn-in documentation, the program 
allows printing a completed DD 1348-lA, shipping paper, required 
DEMIE certifications, and bar codes the DD I348-IA. One of the benefits 
of using ETID is that it allows the disposition sites to review the turn in 
paperwork prior to physical movement of the item and any questions can 
be quickly resolved. ETID requires an account to be established with user 
id and password. (DEA DS, 2014b, p. 20) 

In contrast, there are some anomalies in the documentation’s accuracy when is 
handwritten. The DEA DS website is able to properly correct for errors, whereas 
handwritten documents prepared by the customer at the generating activity may not have 
all or accurate information. Nonetheless, DD 1348s are manually corrected, as noted in 
Eigure 15, but the progress is sluggish when items without documentation or marking 
tags, or that are obsolete, are accepted in the facility. According to DEA DS standard 
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operation procedure (SOP) “Disposition Service Customers may turn property in 
manually or electronically via ETID. Customers who manually prepare the DD Form 
1348-1A should be encouraged to use ETID. ETID will simplify and reduce Generator 
time processing items for tum-in” (DEA DS, 2013c, p. 4). 

Figure 13 illustrates two different scenarios: on the left, items are properly 
marked and DD 1348s are enclosed in the packing protector; and items on the right side 
of the figure are items on a pallet without DD 1348 forms. 



Figure 13. Properly Eabeled Items and Unlabeled Items 
Note. The pictures were taken by the researcher in DEA DS at Camp 
Pendleton, California, in July 2014 


DS’ item process faces efficiency challenges and an additional FTE position— 
Verifier, is established. The verifier researches information in order to identify unmarked 
property. Some of the challenges posed are rapidly finding information for obsolete items 
due to their arrival at DS without manufacture information and part number. The 
difficulty with obsolete items is that electronic information is not readily available. 
Proper identification consumes the verifier’s time in searching for the information on the 
Internet or various databases. For example, the verifier at DS at Camp Pendleton 
conducts research on the Internet for items that do not have paperwork or may have only 
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a local stock number (LSN). An LSN stock is an identification code generated by the 
owning unit prior to being turned in to DS. However, the eode is not linked to the actual 
part number, manufacturer or national stock number (NSN) information, because the 
owning unit may not have had that information in the first place. This disparity of 
information leads to adding additional steps in the disposition proeess that can result in 
additional time to the process. On average, the verifier spend about 13 minutes to 
research an item’s information. Figure 14 shows the results for task mean time 
calculation. 


10.000 Tnals Frequency ^ew 9.782 Displayed 



Figure 14. Computer Generated Task Time using Expert Opinion and 

Pearson Tukey Formula 


Figure 14 shows a simulation of the task time, using a Monte Carlo simulation on 
the Crystal Ball (CB) application. The estimate given by the DS specialist verifier was a 
time requirement of 5-10 minutes to finish the task. However, the verifier claims to have 
taken him up 30 minutes. By using the Pearson Tukey formula to take into aceount any 
expert error, and running the Monte Carlo simulation on CB, the findings show that the 
chance of finishing the task in 5-10 minutes is about 8-43%, respeetively, and the chance 
of finishing the task in 30 minutes is about 3.5%. However, CB shows a 61% chance of 
finishing the task in 13 minutes. Therefore, the 13-minute mean time is used for this 
analysis. The lognormal curve gives the probability of finishing the task at any particular 

54 





















time. According to the Oracle® Crystal Ball’s guidebook (Oracle® Crystal Ball, Fusion 
Edition, [Version 11.1.1.1.00], n.d.), the results in the diagram are interpreted as follow: 
the forecast chart (the x-axis value that is the mean) frequency of about 280, meaning that 
the interval expressed by that column contains 280 values. The mean has a probability of 
less than 0.03 (or 3%), meaning that there is a 3% chance of a value falling within this 
interval. The certainty range includes all values between -Infinity and +Infinity. The 
certainty level is 100%. The display range excludes only one trial out of the total 10,000. 

The DS at Camp Pendleton has currently one verifier who conducts the research 
of identifying items. However, management acknowledges that utilizing an additional 
employee could alleviate the overwhelmed single verifier. One verifier is a single point of 
failure. For example, in the absence of the verifier no one can pick up and continue to 
conduct the research because the other employees are not trained and do not have access 
to the password-enabled databases. Therefore, the verifier is the only one who has been 
granted access to the commercial website. Haystack. This website is a contracted web 
search engine for a limited number of users across DS. Nonetheless, researching 
unidentified items for proper decision disposal is a way to reduce a risk. Safety is always 
considered when getting rid of items so militarized items do not end up in the enemy’s 
hands or exposing hazard to the public. DLA’s core mission is to “protect the public.” 
Figure 15 displays the Haystack on a computer screen. The verifier finds and writes the 
disposal information that was missing on the DD 1348 using Haystack. The verifier 
claims to spend an average time of 10-15 and up to 30 minutes researching items 
consuming a half day’s work. So, for the purpose of this research the 13-minute task time 
per item is utilized, consistent with the computer-generated forecast model. 


55 




Figure 15. Filled Disposal Turned-in Document with Local Stock Number 
Information and Reached on Haystack website 
Note. The picture was taken by the researcher in DLA DS at Camp 
Pendleton, California, in July 2014 


Figure 15 displays a DD 1348 with the printed LSN, but the handwritten 
information was researched by the verifier in Haystack. Figure 15 illustrates the paper 
DTID, with the item’s LSN typed the information was researched in the Haystack web- 
based contracted program. 

Due to their nature, disposed military items that may pose some risk to the public. 
The effect is that it has cultivated a risk adverse culture, and management would much 
rather invest time identifying items rather than selling them as is. According to the 
Receiving SOP (2014c), items are “challenged,” per se, for further investigation and 
verifiers utilize all available resources. For example, the verifier encounters the following 
situation and follows this procedure: 

When to challenge—the first test of whether the DEMIL code of an item 
should be challenged is made by comparing the DEMIL code on the DTID 
[DD Eorm 1348] for the item and the item itself to the DEMIL code in 
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Distribution Standard System (DSS), which comes from FLIS. If these are 
not in agreement, the DEMIL code may need to be challenged. Research 
must be performed to determine what the correct code should be according 
to the established guidelines in the DEMIL Manual. (DLA DS, 2014c, p.7) 

Selling an unidentified item without disposition instructions could potentially be unsafe 
to the public. Eurthermore, this process ensures that items do not end up being reused as 
weapons by adversaries. Therefore, management ensures high effort is spent researching 
the items (Inspector General of the Department of Defense, 2013). 

With a risk adverse-culture, employees exhaust their means to ensure that they 
have disposition codes. However, this creates a drawback regarding time allocated to one 
item. Excess time spent on one item can be a contributing factor to creating a backlog. 
Another drawback is that, within the levels of security measures, items end up in the 
scrap pile for disposal and miss an opportunity for resale. These measures are taken for 
safety reasons. Nevertheless, slow and inaccurate identification reduces timeliness of an 
item’s disposal and the opportunity to continuously process more items. The ratio of Lis 
researched in a week compared to items received is calculated by the following formula: 

Ratio of ESN items researched in a month compared to total items received = Total 
LSN’s researched (sum ESN in Eigure 6, see Chapter III) / average number of receipts for 
the month (month sum of receipts from Eigure 7, see Chapter III) 

Therefore, information from Eigure 6 and Eigure 7 in Chapter III is used to estimate the 
average amount of items and time spent for the month of July researching: 

0.0024= 17/7196 

In other words, less than 1% of accepted items in July were researched. Despite the low 
percentage, it took about 3.6 hours to investigate the information in order to properly 
dispose of the 17 items. 


221 min =17 items X 13 minutes 

Conversely, the verifier downgrades to scrap the items when too much time is 


spent. This generates a missed opportunity because the items may actually have some 
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recoverable value. Moreover, the opportunity cost is in the form that DLA DS either 
decides to pay a contractor to scrap the items that may still have some recoverable 
monetary value if sold in the commercial sector, or incurs holding costs when items are 
stored in warehouses. The benefit of identifying all items upon arrival at DS provides a 
leverage point to reuse or resale more items rather than destroying them. Thus, reselling 
items returns revenues to the DOD. 

b. Problems that Create Backlog 

In the previous section, two factors that affect efficiency were noted: improperly 
completed DD Forms 1348’s, and finding information for obsolete or untagged items that 
promotes excess time spent researching item’s information. One other issue the 
researcher observed during the visit to DS at Camp Pendleton is that the facility receives 
more items than they can process in one day. The backlog accumulates due to the number 
of issues such as cross-dock trucks that arrive each day, customers arrive with 
unscheduled appointments exceeding demand for the day and the demand of items for 
disposal exceed the facility’s capacity to process items. 

A cross-dock truck acceptance is property from another DLA DS site. The other 
site does not have the capability to process items due to its limited authority to coordinate 
with contractors or because it processes only certain types of items, thereby transferring, 
or cross docking, it to DS at Camp Pendleton. Items outside its scope are delivered to 
bigger facilities that have greater authority to dispose of items. Nevertheless, on a daily 
basis, the Camp Pendleton site accepts ten trucks from local customers and an average of 
three cross-dock trucks from other sites. Each truck is assigned an RCN that tracks the 
number of days it takes to process the pallets and the estimated number of Lis in them. 
This observation was confirmed with the data provided by the area manager. 
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Figure 16. DLA DS at Camp Pendleton Average RCN (July to October) 


Additionally, RCN become a nuisance such that the FTE may underestimate how 
many items are in the RCN. According to the SOP, the FTE estimates the amount of Lis 
prior to stowing the items in the backlog area: 

A Receipt Control Number (RCN) is an eight character alpha/numeric 
sequence which indicates the Calendar year, month, day, and the number 
of turn-ins that is manually generated via Distribution Standard System 
(DSS). The RCN is created when a conveyance is received on site prior to 
the receipt process of Disposal Turn In Documents (DTID). The RCN 
provides workload visibility by identifying estimated line items received 
from individual turn-in customers, a pallet count, and a date and time the 
property was accepted. The sum of all active RCN line counts will 
determine and report Disposition Sites estimated unprocessed receipts. 

(DLA DS, 2013a, p. 1) 

The RCN is the first control number tagged to the pallets in each truck conveyance 
however a pallet does not have a limit on how many Lis it is required to contain. One 
pallet could have one big bulky item, or it has an average four line items. The data shows 
that an RCN has an average of 13 pallets resulting in an average of 50 items per RCN. On 
average the DS at Camp Pendleton can process about 300 items per day with the current 
staff capacity, and on average the facility received 8.5 RCN per day resulting about 425 
items to be processed per day. This study covered almost a four-month period, and during 
that time there were on average 2,259 items that accounted for about 47 RCN or 7.5 day’s 
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worth of backlog that was in the backlog stowage area, suggesting that each RCN had 
almost 50 LI, as depicted in Figure 16. Therefore, the remaining items that are not 
processed daily go to the backlog for future processing. The coefficients are explained 
with the formula: 

125 average daily of items for the backlog stowage area = 425 - 300 

At the beginning of the period of study the oldest RCN was 30 days old and at the 
end of the study period, was 45 days. This suggests that the ongoing demand surpasses 
capacity to process items for disposal. The logic of ongoing demand exceeds the process 
is because the items continuously arrive so items that arrive are processed plus the items 
in the backlog, the ongoing issue mitigates to reduce the backlog. However, if the crew 
proposes only items from the backlog without receiving any RCN it would take about 
7.5 days. Calculated by 2,256 average items in backlog / 309 average items DS at Camp 
Pendleton can process as day. 

At the beginning of July, the facility started with 1,082 line items in backlog. By 
the end of July, the facility ended with 3,072 Lis in backlog and accepted a daily average 
of 313 items. How did it triple? The disparity is noticeable in the first 7 days of July. On 
July 1-3, the facility accepted about 14 RCN’s and the backlog doubled, from 1,082 to 
2,320, suggesting that each RCN had an average of 51 Lis. Additionally, the staff could 
only completely process about 600 items in the 2-day period, so the remainder that was 
not processed was kept in the backlog. This scenario is ongoing through the days and 
months so the backlog does not reduce to zero, as illustrated in Figure 17. Furthermore, 
for the month of July an upward trend illustrates demand exceeded than the amount of 
items that were processed. 

However, accepting Lis is not the only task in the focus of effort. The warehouse 
duties include storing, inventory, shipping, customer pick-up, rearrangement of backlog, 
cleaning, and scrap management. In addition, the site provides property disposal classes 
to all local units and accepts the property from four other cross-dock sites for processing. 
Considering all these factors suggests a growing problem that has potential leading 

bottleneck, backlogs, and increased disposition time. 
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Days' Worth of Backlog for July 



Day 


Figure 17. DLA DS at Camp Pendleton Day’s Worth of Backlog Items for July 2014 

Figure 17 depicts all DS working days for days’ worth of backlog items for July. 
The calculation is the result of using the backlog ratio formula such that the calculation 
for 1 July and so on is the result from this computation: 

2.3 days’ worth of backlog items = (1082 backlog items / 486 items processed on 1 July) 

The backlog ratio steadily grows throughout the month of July, reaching an 
average of 6.9 days’ worth of items in the backlog for the month, depicted in red above 
number 32. Equally, day 23 is a day of concern because it reached over 22.5 days’ worth 
of working items from the backlog stowage area, suggesting that a double amount of 
items were received compared to the previous day. Nonetheless, for the four-month 
period, the trend is downward, as illustrated in Figure 18, resulting in an average of 7.5 
day’s worth of backlog items. Arguably, this numbers suggest a backlog of days worth of 
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items for proper disposition is ongoing and management needs to find a solution to 
reduce it and eliminate backlogs. 


Days' Worth of Backlog 



Day 

Figure 18. DLA DS at Camp Pendleton Day’s Worth of Backlog Items 

from July to October 2014 


Inverse trend relationship between processed versus backlog items. The data 
shows a trend that showcases underlying indicators of an inverse effect that an increase of 
backlogged items results in the decrease of processed items and vice versa. The trend is 
observed when at the beginning of the week more items are accepted affecting items 
processed to fewer numbers. Conversely, fewer items are accepted thus creating less 
backlog and more items are processed thus increasing the amount of processed items. 
This inverse relationship is clearly depicted in Figure 19, suggesting that in order to 
reduce the backlog fewer items must be accepted or productivity must be increased to 
equal out both components. Additionally, the backlog ratio compared to the number of 
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items processed is almost triple, indicating that DS at Camp Pendleton cannot keep up 
with more demand than what they can process. DS’s property acceptance process is not 
enabled with technology and automation, so it does not deliver efficient or effective 
outcomes when benchmarked with comparable commercial practices. 
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Inverse Relationship between Backlog and Processed 

Items 


4,000 *Normalized data treated blanks as zeros because 
DLA DS does not work on the weekends 

3.M0 


Frequency (weekly from July to October) 


Backlog items 
decreased 
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Figure 19. DLA DS at Camp Pendleton Inverse Relationship Pattern between 
Backlog and Processed Items from July to October 2014 
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Figure 20. Two Military Tents with Backlog Items 
Note. The pictures were taken by the researcher in DLA DS at Camp 
Pendleton, California, in July 2014 


Figure 20 shows two pictures of two dome tents outside the warehouse at DLA 
DS, Camp Pendleton. When the picture was taken there were about 47 RCNs with an 
estimated 2,313 items that accounted for about 7.5-backlog ratio in days’ worth of items 
that are awaiting disposal. 

c. Analyzing Backlog using Analytical Model to Resolve Backlog 

A backlog of items that are waiting for future processing is a phenomenon created 
by unprocessed items that have been received but are not processed or are in the process 
of being completed because the number of items arrival exceed the employees’ capacity 
to process. Unpredictability of items arrival and variability of service processing time 
leads to backlog or queues. Understanding and learning the queuing is an important area 
in operations management (Jacobs & Chase, 2010). For DLA DS at Camp Pendleton, 
item-processing demand vary by the amount of items processed per employee and often 
the amount of items exceeds normal capacity that the six-crew employee at Camp 
Pendleton can handle. However, the arrival of items can be controlled in a variety of 
ways. For example, DLA DS can reduce the amount of appointments per day, thereby 
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reducing the amount of items accepted, they can limit the number of items per pallet so 
that they only receive about 300 items per day, or extend the hours employees work but 
only process items in the queue and not accept more items during the extended hours. For 
the employees, DLA DS can affect service time by adding more employees, or 
implement machines to automate the process flow to increase productivity. 

According to Jacobs and Chase (2010), the queuing system consists of three 
components: (1) the source of items and the way items arrive, (2) the servicing system 
such as servers or employees, and (3) the condition the items exit the system (p. 40). By 
calculating the three queuing components one can formulate the number of employees 
DLA DS at Camp Pendleton are needed to reduce and prevent days’ worth of backlog. 
Therefore, to formulate the amount of employees are needed at DLA DS the coefficients 
in the following formula were utilized in a spreadsheet. Figure 21, designed to compute 
the characteristics of employees in the multiple-server queuing model using arrival rate of 
items and service rates each employee processes. 

The queuing model illustrated in Figure 21 used results from previous section 
with regard to averages of items received at DLA DS at Camp Pendleton, California, 
average processed items per employee per hour, and number of employees in the items 
disposal crew at DLA DS Camp Pendleton, California. That means that on a daily basis 
the Camp Pendleton facility received an average of 425 items for disposal; therefore per 
hour they accepted about 65 items per hour and an employee can process 7.5 items, 
suggested by the DLA DS study (2013) and the Camp Pendleton facility (2014) had a 
six-employee crew for item disposal during the period of the study. 
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Analysis: Number of Employee Required at DLA DS, 


Camp Pendleton, CA 


Assumtions: Multiple servers, Infinite population, Poisson arrival. Exponential service time. 

Unlimited waiting warehouse size 



Yellow cells need user inputed values 


Inputs 

Coefficient unit: 


Unit of time 

Hour 



Arrival rate (lambda) 

65.38461538 

Items per hour 


Service rate (mu) 

7.5 

Items per hour 


Number of identical employees (server [s]) 

9 

Employees 



Summary measures 

Notes: 


Average utilization rate of server 

96.9% 

Capacity 

Average number of items waiting in backlog (Lq) 

27.64978 

Items 

Average number of items in system (L) 

36.36773 

Items 

Average time waiting in line (Wq) 

0.42288 

Hour 

Average time in system (W) 

0.55621 

Hour 

Probability of no customers in system (PO) 

0.00003 

Probability of 
empty system 

Probability that all servers are busy 

89.5% 

Percentage of 
items that wait in 
backlog 

Probability that at least one server is idle 

10.5% 

Percentage of 
items that don't 
wait in backlog 


Figure 21. Spreadsheet of Analysis the Number of Employees Required at DLA DS 

in Camp Pendleton, California 


Figure 21 is an Excel template for using analysis with the queuing model to 
choose the number of employees for DEA DS at Camp Pendleton, California. The 
spreadsheet was designed to compute characteristics of multiple employees, using the 
average arrival rate of items per hour, and service rate each employee can process per 
hour. The output coefficients are the mean time of items arrival at per hour during an 
expected 6.5-hour working per day, and the mean time items processed for a crew of 
employees per hour during an expected 6.5-hour per day. 

Table 4 illustrates the results from the queuing model with changing employees 

from six up to ten. With eight employees one can see significant improvement to the 
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probability to decrease backlog from 255% to 135%. Adding the ninth employee 
provides di 91% capacity meaning that the facility can keep up with demand by 
processing all received items and have a 3% idle of down time, which will be initially 
used to reduce the 7.5 days’ worth of backlog. Therefore, with nine employees the 
facility provides a 97% capacity in term of processing items upon arrival to mitigate 
backlog in the future. On average DLA DS received 425 items a day and with a crew of 
six the DLA DS at Camp Pendleton processed about 300 items; thus, on a daily basis 
about 125 items are placed in the backlog to process them on the first-in first-out basis. 
The facility had 7.5 days’ worth of backlogs; therefore with nine employees the facility 
will slowly reduce the backlog and be at capacity with relations to items arrival, because 
the facility will be able to process 450 items per day and about 425 items are accepted. 


Table 4. Sensitivity Analysis of Number of Employees to Reduce Backlog 


Sensitivity Analysis: Number of Employee 
Required at DLA DS, Camp Pendleton, CA 

Employee (s) 

Facility 

Capacity 

Percent Items 
in Backlog 

Percent Items 
not in Backlog 

6 

145% 

255% 

- 

7 

125% 

185% 

- 

8 

109% 

131% 

- 

9 

97% 

90% 

10% 

10 

87% 

59% 

41% 


d. Summary 

The process flow is very lengthy, and DLA DS needs to manage it in an average 
of 42 days in order to provide an opportunity to RTD items to branches of service, federal 
agency, or local agency. DLA DS needs reduce backlogs, increase efficiency and take 
advantage of opportunities. As seen in this analysis, there are steps in the process flow 
that have potential to create a bottleneck. This section examined the three steps in the 
items process that are areas of concern. The results show that there are four areas of 
concern that contribute to creating days’ worth of backlog items. Lirst, improperly filled 
DD 1348 provides for additional time to spend conducting research in order to fill the DD 
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1348 for further processing. Second, verifiers spend time to conduct research on items 
that are not labeled or documented on a DD 1348. About less than 1% of accepted items 
at are unlabeled. However, the 1% of unlabeled items is not the main problem of days’ 
worth of backlog items, the third problem can be safely argued that DLA DS at Camp 
Pendleton accepts more items than what it processes, resulting in an average over 2000 
items in the backlog stowage area. DLA DS at Camp Pendleton is limited to processing 
about 300 items per day but it received an average 425 per day in an average of 8.5 RCNs 
per day, leaving the remainder to add to the backlog; therefore, the demand exceeded 
the amount of items that were processed. Similarly, fourth, RCNs are not limited to the 
amount of items that will fit in a pallet or tri-box, such that an RCN can have about 
50 items. The results from the queuing theory suggests that, with nine employees, the 
DLA DS at Camp Pendleton will keep up with demand by processing all received items 
and have a 3% idle time, which will be utilized to reduce the 7.5 days’ worth of backlog. 

DLA DS made progress in reducing the lead-time on the process flow by 
implementing the Lean Six Sigma in 2008; however, additional changes need to be 
created in order to improve the lead-time. In general, these challenges are corrected with 
adequate automated processes and the focus of this study is addressing inefficiency to 
correct it with the implementation of technology. With that in mind, the next section 
assesses object/shape recognition technology COTS sectors. 

B. ASSESSMENT OF OBJECT/SHAPE RECOGNITION TECHNOLOGY 

MATURITY 

I. Overview 

The Technology Readiness Assessment (TRA) is designed to allow program 
managers, acquisitions personnel, and decision-makers to understand the risk, cost, and 
time associated with maturing technologies. This section of this study reports the results 
of the assessment results of three different companies’ object/shape recognition 
technology. The three companies of interest are listed in Figure 22 with each company’s 
description. The assessment follows the Department of Defense Technology Readiness 
Assessment Guidance (ASD[R&E], 2011), and for the purpose of this study the 
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researcher adopted and customized the Department of Energy’s (2012) template for 
assessment to make it suitable for assessing object/shape recognition technology systems. 
The assessment consists of separate reports for each companies’ technology that include 
(1) an overview of the company’s object/shape recognition technology profile; (2) 
technology’s goal; (3) benefits; (4) critical area(s) of challenge; (5) qualitative summary 
of the TRL; and (6) a statement of relevancy to DLA DS. Finally, the section concludes 
with a summary of the strengths and weaknesses of object/shape recognition technology. 
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Object Shape Recognition Companies and their 

Programs 


Cognex delivers the widest range of image-based factory automation and 
distribution products to help companies I ike you ensure error-free 
production, lower costs, and manage your supply chain; 

Barcode readers to ensure part and product traceability 

Vision systems and software for quality inspection and automation 

Vision sensors to detect part features and verify assembly 

Laser profilers and software for 30 inspection to further optimize quality 

Source; www.cognex.com 


COG 


NEX 


KmierWiJty^ 






When business leaders and innovators like you engage with Imaginestics and 
discover the VizSeek® shape-search engine, their first question is often; 

"Can it do this?" InstoKth-ely, the answer is always, "Yes." 


The VizSpace supply-chain sourcing network and the evolution of clientele 
licensing our shape-search technology are an prime examples of true 
American ingenuity. If you can ensision a furiction for shape search, inside 
the VizSpace network or out, the Imaginestics development team is available 
to partner with you and engineer a new ^Tplication that meets your group's 
unique requirements. 


We have a creath'e, technicany innovative team adept at taking your 
imagined solution and making it a reality. The possibilities for shape search 
are limitless — firom determining cost to configuring products. Get in touch 
with us and give us your idea. The possibilities are l^tless! 

Source; www.imaginestics.com 


Augmented Reality 

RMD isdevelopinga revolutionarytrainingandenablingtechnology based on 
augmented reality. It isextremely valuable for speeding training and repairs, 
detecting and preventing errors, saving money and time, and improving 
safety. 

Source; www.rmdinc.com 


Figure 22. Three Companies with Recognition Technology Programs 
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2. Cognex 

Overview: Cognex’s technology is a belt-driven single file carousel that identifies 
items by scanning bar code tags. The items are associated with a database mainly by 
using the serial numbers on the item’s tag. Cognex manufactures the hardware like the 
image capture devices and they partner up with companies that offer integrated solutions 
to connect Cognex hardware. However, for large multi-million-dollar project Cognex 
offers direct integration to innovate a system. Consequently, Cognex does not have a 
prototype application to qualify for DLA DS requirement, but its current concept of 
processing items is applicable to DLA DS. 

Cognex Technology’s Goal: Currently, Cognex’s technology is used in 
companies where specific item precision is required. For example Cognex technology 
detects defects with sensors, 3-dimensional (3D) lasers, scanners and cameras in order to 
improve production productivity at a higher speed. 

Benefits: The technology’s approach is to provide efficiency and effectively 
manufacture or inventory items. The industries that benefit from Cognex technology are 
automotive, food and beverage, and electronics as depicted in Figure 23. With regard to 
DLA DS, Cognex’s concept that can be applicable currently is not in a COTS form that 
DLA DS can plug and start to operate. 

Critical Area of Challenge: Critical technology challenges related to meeting 
DLA DS requirements include the fact that the Cognex machines are specific to a 
company’s design and needs, so a prototype to fit DLA DS needs will need to be 
reengineered from the ground up. What this means is that operational equipment intended 
to identify an object without bar codes technique is not available at Cognex. However, 
Cognex has the technology and partners to conduct R&D in order to custom make a 
technology for DLA DS. 

Qualitative Summary of Cognex’s Technology (TRL): A technology that is 
ready with DS’ specification and to be operated at DS facility or similar environment has 
not been created. Clearly, Cognex will be required to engineer and integrate elements 
from partners that would require technology maturation; therefore, Cognex’s technology 
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values in the TRL scale 2 (see Table 5 for the TRL definition). Currently, Cognex has 
made machines that operate at specific owners requirement, so they have the concept and 
principle formulated relevant to DLA DS requirement. However, technology for DLA DS 
would be considered to be in a limited paper study such that its current system would be 
left for speculation to be applicable as it is; therefore, further maturation would be 
required under an investment program and contract. 


Table 5. Description of Technology Readiness Level 2 (from USAF, 2010) 


TRL 

Definition 

Description 

2 

Technology Concept and/or 
Application Formulated 

Invention begins. Once basic principles are 
observed, practical applications can be invented. 
The application is speculative and there is no proof 
or detailed analysis to support the assumption. 
Examples remain limited to paper studies. 


Statement of Relevancy to DLA DS: Cognex explicitly described that a project of 
large scale, such as full automation with carousel belts, scanners, and image capture 
devices, would be worth pursuing. Meanwhile, a small project, such as a mobile device 
with image capture apparatus, would be an undertaking for one of Cognex’s partners and 
eventually Cognex would mature it to a grand scale. 

Figure 23 illustrates Cognex technology that is designed to scan and read bar 
codes while the items move on a carousel system. Similarly, the laser and scanner can 
detect anomalies and read codes and discrepancies with items specifications. 
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DataMan Logistics Barcode Reader Applications 



Tote Scanning Single-sided Scanning Carton Coding Print & Apply 



Figure 23. Cognex Product Identification Machines 
Note. Picture from Cognex Pamphlet and Cognex YouTube video 


Figure 24 illustrates a large-scale supply chain logistics warehouse operated with 
Cognex technology. The technologies included are scanners, and laser and image capture 
devices to efficiently and effectively control inbound and outbound inventory. 
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©Inbound 


O Order Fulfillment 



Cognex readers have: 

• The highest read rates • Performance feedback • No moving parts 


Figure 24. Cognex Large Scale Machines (from Product Cognex, n.d.) 

3. Imaginestics 

Overview: Imaginestics has developed a technology that is an application for 
mobile devices, which includes phones and tablets for both Apple and Android operating 
systems. The technology is designed to find shapes and objects that are similar to a given 
visual input, labeled by Imaginestics Smart Shape Technology and trademarked as 
VizSeek®. A picture captured by the mobile device is used to search shapes in a 
VizSeek® database. The image’s shape is associated with an indexed database to provide 
feedback. The mobile device conducts the search on VizSeek® which requires either 
Internet or an intranet (behind the firewall) to access the database. To create the indexed 
database, VizSeek® extracts and archived shapes and text from images, documents, and 
other sources of database. Imaginestics main contribution to the shape recognition 
technology is to extract the information by normalizing the data so that information is 
indexed from the content, such as diagrams and images from manuals, publications, and 
so forth. Imaginestics technology is capable of searching images from two and three- 
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dimensional models (see Figures 15 and 16). This means that it recognizes the image 
from any angle and format rather than recognizing an image to an image. Imaginestics 
was a winner of the Indiana Entrepreneurship Award. The merit of the award is 
outstanding, but what is of importance to this assessment is the fact that the DOD funded 
the research, so Imaginestics understands the DOD acquisitions process. The award 
citation states the following: 

Imaginestics’ vision is to organize and market industrial manufacturer’s 
product and service information and make it visually searchable and useful 
worldwide. As the first step in making industrial manufacturer’s product 
and service information visually searchable...developing 2D shape 
matching technology and unique intranet search engine... [and]... 
developing the 3D shape search algorithm at Purdue University. [Then]... 
developed VizSeek, the world’s first online visual search engine for 
manufacturing. In addition, they also worked on value added solutions 
such as VizAdvise, VizCompare and VizConfig to make the search results 
obtained useful for the search user. Imaginestics received funding support 
from National Science Foundation, Department of Defense and Indiana 
Economic Development Corp. to conduct research and development of 
these solutions. (“Indiana Entrepreneurial Eeadership Awards Winners,” 
n.d.) 

Imaginestics Technology’s Goal: Imaginestics’ technology is used to organized 
information such as shapes of objects so that it is searchable using a mobile device and 
feedback with the images’ information is provided in seconds under the condition that the 
database has the information indexed. 

Benefits: The technology’s approach is to provide information from an image. 
The industries that benefit from Imaginestics’ technology are aviation and supply chain 
management. Currently, Eockheed Martin uses the technology to conduct research and 
make costing estimates. With regard to DEA DS, Imaginestics’ technology is applicable. 
As a COTS technology DEA DS will require further maturation to be applicable to DEA 
DS requirement. For example Imaginestics is capable of indexing information from 
converted images in optical character recognition (OCR) format such as printed manual 
in order to digitized it and index it in the VizSeek® database. This modification will 
require the buildup of a database specifically for DEA DS. Another benefit from 
Imaginestics is that the mobile device that captures the image can be used to 


75 



consequently upload the same picture in DLA DS liquidation website to resale the items 
rather than contracting a photographer. 

Critical Area of Challenge: Critical technology challenges related to meeting 
DLA DS requirements include the fact that a database will have to be built. Additionally, 
because the system operates via the Internet and Apple Inc. devices the challenge relates 
to getting approval from the Defense Information System Agency (DISA) with respect to 
information assurance (lA) credentials. However, the credibility package is part of the 
acquisitions process. 

Qualitative Summary of Imaginestics’ Technology (TRL): Imaginestics 
technology is the closest technology that is ready with DS specification and to be 
operated at a DS facility, however is has not been operated in a DS similar environment 
under DISA lA requirements. Imaginestics’ shape recognition technology is being 
utilized by Lockheed Martin, so technology components relevant to DLA DS requirement 
is known to work. However, the database aspect of the technology will be required to be 
engineered and to integrate elements from the current prototype, which would require 
maturation; therefore, Imaginestics’ technology values in the TRL scale between 4 and 5 
(see Table 6 for the TRL definitions). Currently, Imaginestics has made machines that 
operate at specific owner’s requirement, so it has the concept, principle, and prototype 
formulated relevant to DLA DS requirement. The system hardware and software 
integration has progressed beyond the laboratory phase. 
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Table 6. Description of Technology Readiness Levels 4 and 5 

(from USAF, 2010) 


TRL 

Definition 

Description 

4 

Component 

and/or 

Breadboard 
Validated in 
Laboratory 
Environment 

Basic technological components are integrated to establish 
the feasibility that the pieces will work together. This is 
relatively low fidelity compared to the eventual system. 
Examples include integration of ad hoc hardware in a 
laboratory. 

5 

Component 

and/or 

Breadboard 
Validated in 
Relevant 
Environment 

Eidelity of breadboard technology increases significantly. 

The basic technological components are integrated with 
reasonably realistic supporting elements so that the 
technology can be tested in a simulated environment. 
Examples include high-fidelity laboratory integration of 
components. 


Statement of Relevancy to DIA DS: Imaginestics has been approved for lA 
approval for another technology that is contracted under the Navy and Office of the 
Secretary of Defense (OSD); therefore Imaginestics understands the DOD acquisitions 
processes and lA approval request process. 

Figures 25 and 26 illustrate the search and results of a compressor, respectively. 
The image of the compressor in Figure 25 was matched with an image of a compressor 
from government liquidation website. The results illustrated in Figure 26 is the interface 
from VizSeek® depicting the feedback information of the compressor, such as the cost, 
NSN, DMIL code and DTID number related to the order tracking number entered by 
DLA DS when the compressor was processed for resale. 
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Search Results-Compressor, Reciprocating 



No CTurtstek 



A photo of a compressor is used to search and find matching DLA compressors. 


Figure 25. VizSeek® Application Matches Compressor Image with Database 
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Figure 26. VizSeek® Results with Image and Pertinent Information 
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4. 


RMD 


Overview: RMD created a technology, called the automated digital recognition 
technology (ADIRT), which is focused on the artificial intelligence perspective. The 
technology is designed to capture images but the user verifies the information the first 
time the image is introduced to the ADIRT. The subsequent time when the image is 
presented ADIRT automatically recognizes it to processes it for identification and 
provides feedback. RMD built a prototype for the Marine Corps. The Marine Corps 
initially funded the R&D; however, under the budgetary cuts the program was no longer 
funded. Consequently, RMD has not matured the technology any further; however, 
RMD’s technology qualifies within the DLA DS requirement. RMD’s technology 
concept of processing items is applicable to DLA DS, but requires further funding for 
maturation. RMD estimates a two-year period to provide a prototype to DLA DS. 

RMD Technology’s Goal: ADIRT under its current state was designed to support 
electronic technicians to automatically recognize a circuit board retrieving schematics for 
repairs. ADIRT uses a laser projector to point at the part in the circuit board so the 
mechanic conducts the repairs to the component in the circuit board. This enables the 
technician to identify and locate unlabeled components and part numbers easily, correlate 
them with documentation, and communicate with databases across platforms and 
organizations (RMD, 2014). 

Benefits: The technology’s approach is to capture knowledge in the form of 
artificial intelligence so that any technician uses the technology to guide them when 
repairing the circuit board. The Marine Corps concept was to retain in a database the 
knowledge of well-training electronic technician and pass on the knowledge to any junior 
Marine. Given that ADIRT is designed to remember procedure in accordance with 
technical manuals, when the technician fixes a circuit board, ADIRT provides the 
potential to capture any knowledge to turnover for continuity. 

Critical Area of Challenge: Critical technology challenges related to meeting 
DLA DS requirements include the fact that the initial prototype was built for a different 
purpose. Additionally because the system operates via the Internet and has the capability 
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to operate in a mobile device, the challenge relates to getting approval from the DISA 
with respect to lA credentials. However, the credibility package is part to acquisitions 
process. Additionally the current prototype is not in a mobile device state. The current 
prototype is a computer with a camera and laser projector, so the challenge is to reduce 
the size to a mobile device, which calls for further maturation. 

Qualitative Summary of RMD’s Technology (TRL): RMD will be required to 
engineering and integrating elements from its current prototype, ADIRT, which would 
require further R&D for the technology to mature; therefore, RDM’s technology values 
in the TRL scale 2 (see Table 7 for the TRL definition). Currently, RDM has made a 
prototype that operated for the Marine Corps requirement of training technicians to 
conduct repairs, and the technology detects and prevents errors. RMD has the concept 
and principle formulated relevant to the DLA DS requirement. However, to make RMD’s 
technology fully relevant to DLA DS, the technology is considered to be in a limited 
paper study such that RMD current system is beyond speculation to be applicable as it is; 
therefore, further maturation would be required under an investment program and 
contract. 


Table 7. Description of Technology Readiness Levels 2 

(from USAF, 2010) 


TRL 

Definition 

Description 

2 

Technology Concept and/or 
Application Formulated 

Invention begins. Once basic principles are 
observed, practical applications can be invented. 
The application is speculative and there is no 
proof or detailed analysis to support the 
assumption. Examples remain limited to paper 
studies. 


Statement of Relevancy to DLA DS: RMD explicitly explained that the 
technology was designed for mechanical maintenance rather than for supply-chain 
management. According to RMD it will 

deliver a prototype to DLA in the tenth month of year one for testing at 
their facilities. In year two of the program, RMD will deliver a more 
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advanced prototype in month 22 of the 24-month program for test and 
validation at the DLA” (RMD, 2014). 

Therefore, RDM’s technology is not ready for the near future and would required 
excess funding to mature it. 

Figure 27 illustrates RMD’s technology that is designed to scan and read 
schematics while the technician makes repairs to the circuit board. The laser projector is 
designed to pinpoint which component on the circuit board to repair. The technology is 
designed to learn the process the first time it is introduced with a circuit board. The 
subsequent time when the same-type of circuit board it repaired the system navigates the 
technician to conducts the repair to the component as previously conducted. 



Figure 27. RMD Product Identification Machines (picture from rmdinc.com) 

5. Strengths and Weakness of Object/Shape Recognition Technology 

The root causes for automating a company with technology is to cut costs, target 
inefficiency, and control breakdowns in managing inventory, and capture data for 
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analytics purpose. Technology presents strength and weakness, including strengths with 
standardization, productivity, inventory control, and management. Conversely, 
technology presents weaknesses with upfront capital investment, reliance on the 
information and information assurance, and equipment fragility. 

a. Strengths 

1. Standardization—technology ensures uniformity and conformation across 
the board. 

2. Productivity— technology can reduce time in identifying items, thereby 
increasing productivity. 

3. Benefit—the benefits will outweigh the initial investment cost, in the long 
run, usually after the payback period then benefit grows by providing time 
and labor saving earnings. 

4. Accuracy—error is marginal, when proper information is indexed to return 
correct feedback. A database program provides the ability to conduct data 
mining for analytics. Analytics improves operational efficiency by 
facilitating effectiveness of processing items. 

5. Operate—ease of operation with minimal training. 

b. Weaknesses 

1. Cost—initial upfront investment cost and maintenance cost. Additionally, 
technology has a double-edged sword, such that favorable benefits are 
gained upon the recovery of the funds expended. However, the technology 
depreciates and wears to the point of affecting its performance— 
ultimately becoming inoperable. Therefore, replacing it adds another 
round of cost. 

2. Feedback—reliance on information keyed in the database. In other words, 
the feedback accuracy relies on the manually entered information in the 
database, unless it is an artificial intelligence that is adaptive and self- 
learning. Nonetheless, the feedback could be unreliable due to incorrect 
data in the database. The trend is counteracted if the wrong information is 
entered where the focus should be elsewhere. 

3. Fragile—environmental conditions or accidental mishaps may break the 
equipment. 
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C. CHAPTER SUMMARY 

This chapter presented the examination of the data and item process from DLA 
DS at Camp Pendleton, California, as well as an assessment of three companies’ 
object/shape recognition technology. Based on the analysis of item process, the data 
suggests and emphasizes that there are four areas of concern that drive inefficiency and 
backlog. Improperly filled DD 1348 added time to the items disposal process due to 
researching the item’s information; RCN’s items are limitless and backlogs are created 
due to accepting greater items than it can be processed qualify for factors that require a 
solution and portrays the importance of this study. 

Despite previous changes to streamline the steps, the current process can be 
further improved to reduce or eliminate backlogs. Generally, automation though use of 
technology generates favorable conditions that can be considered for implementation. 
Currently, the private and government sectors have some form of object/shape 
recognition technology to support their missions. Of the three companies’ technologies 
that were assessed under the TRA guidance, Imaginestics technology is the most viable 
and mature system, ranking up to a TRL 5. However, the standard dictates that only a 
TRL 7 or above is mature, and therefore, can be considered for implementation. 
Imaginestics technology has the potential to be a quickly working model and ready for 
implementation with less considerable time and cost, compared to the other two 
technologies analyzed here. An object/shape recognition tool offers benefits, such as 
reducing property identification time, when a well-designed infrastructure is in place. 
However, consideration of upfront cost must be analyzed to justify the investment and 
determine whether it will be sustainable in the long run. The focus of Chapter V is the 
CBA that presents financial aspect of investing in object/shape recognition technology. 
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V. COST-BENEFIT ANALYSIS 


The objective of this chapter is to examine whether the expected benefits of 
implementing object/shape recognition technology justify the cost. The analysis generates 
a measure of the expected economic return on investment, and evaluates the feasibility 
that object/shape recognition technology can be installed at DLA DS at Camp Pendleton. 
The CBA is consistent with the U.S. government guidance: Circular A-94 Guidelines and 
Discount Rates for Benefit-Cost Analysis of Federal Programs (0MB, 1992), and DOD 
Directive 5000.2 Interim—supportability analysis (DOD, 2013a). The benefits are 
estimated as cost-savings from reduction in time to dispose items resulted from 
implementing object/shape recognition technology to the item disposal process. 

This CBA analysis evaluates two competing alternatives: the status quo, which is 
labor intensive, and the option of investing in object/shape recognition technology for the 
purpose of replacing the status quo. This study is an ex ante CBA-type conducted prior to 
the project in order to show whether it is worth investing in object/shape recognition 
technology and to provide information to DLA DS decision-maker prior to making the 
investment. Notwithstanding, there are nine basic steps in conducting a CBA, described 
as follows: 

Step 1: Specify the set of alternative projects. DLA DS currently operates the 
property process without technology; therefore, this alternate is considered status quo and 
maintains DLA DS property identification in its current state. The second alternate will 
be evaluates and analyzes operations with technology. 

Step 2: Decide whose benefits and costs count (standing). The stakeholders are 
identified as follow: DLA DS decision-makers consider the cost and the benefit of the 
alternatives. DLA DS employees are directly impacted by the decisions because they 
conduct the property identification process. The U.S. Congress mandated that DLA DS to 
cut operating costs, and taxpayers continue to fund the U.S. government through taxes. 
This thesis focuses on providing information to DAL DS decision-maker. 
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Step 3: Catalog the impacts and select measurement indicators. The analysis 
result includes input of time in units and cost of labor. Comparably, the focus is then 
shifted to associate cost of labor with and without technology in order to measure cause 
and affect outcomes. The results should provide benefit and cost impact. 

Step 4 : Predict the impacts quantitatively over the life of the project. The time 
horizon is limited to 10 years to discount the cost of technology depreciation and 
replacement. There is uncertainty beyond the 10-year period, and to capturing that 
prediction is complex. 

Step 5: Monetize (attach dollar values to) all impacts. 

Time as a variable is monetized in order to provide the return on investment and 
to make the baseline for the “willingness to pay.” The researcher determined that 
investing in object/shape recognition technology would not incur revenues, and the 
investment is for only one facility, thereby not being a big acquisition program to raise 
revenues. The nature to justify the investment is costs saved. The DLA DS does not make 
revenues, given that it is part of the DOD; rather, the benefits is the money saved by 
reducing time task and increasing productivity. Therefore, the employees’ work-time is 
monetized and considered only when the particular employee uses the technology to 
reduce task time researching items, consequently increasing productivity. 

The researcher calculated the annual wages for a GS-11 in dollars per hours in 
order to monetize the LSN research task, such that the assumption is that automating the 
identification process with technology will save money in labor cost. The Verifier at 
Camp Pendleton is a GS-11 whose annual wages is between $50,790 and $66,027 and 
spends half the day researching items’ information. Therefore, this study calculated the 
following information, as depicted in Table 8. 
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Table 8. The Annual Cost of Researehing Items Is $27,282 


DLA DS Billet: GS 11 Pay 

$50,790.00 - $66,027.00 / Per Year 

For this study $59,255 is utilized for calculations 

Description 

Result 

Formula 

$ per hour 

$28.49 

59255 ann wgs /(8hr day * 5 days)*52wks 

Mins spent on 17 items 

221 

13 min *17 items 

Hrs per day spent researching 

3.68 

221 min/60 in hr 

Cost per day 

$104.93 

$28.49 per hr * 3.68 hrs per day 

Annual cost researching 

$27,281.99 

($105 cost per day*5days) 52 wks 


Table 8 illustrates ealculations that result in $27,282 a year to researeh items’ 
information. The caleulation accounts for a GS-11 Step 6 who dedicates almost four 
hours a day with items that need furthers steps to research information on one of three 
databases to properly dispose the items. The effect of the change in cost is variable due to 
the levels of GS with step wage an employee may hold. Furthermore, the scope of this 
thesis limits the amount of variables one can research and utilize for accounting how any 
other income from other variable may generates more market comparable revenues. 
However, the net present value (NPV) utilizes the final cost of $27,282 as a form of 
benefits. 

Step 6: Discount benefits and costs to obtain present values. This analysis took 
into account a 2.5% social discount for labor pay-raise increases. Additionally, a nominal 
discount rate is used to discount cost and benefit over the 10-year horizon time. Thus, in 
order to account for the time value money this research used a 7% discount rate. 
According to the Circular A-94, when analyzing proposed investment the 7% rate is ideal 
for calculating the return on investment in the private sector (0MB, 1994). 

Step 7: Compute the net present value of each alternative. The NPV equals the 
present value of benefits minus the present value of costs: 

NPV = PV (B) - PV(C) 


Therefore, corresponding with the formula the cost of status quo is in the form of 
the opportunity cost, such that by leaving the item disposal process as is DLA DS misses 
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the opportunity to increase productivity; thus, the alternative with technology reaps the 
benefits in the form of money saved by reducing time task and increasing productivity. 

Nevertheless, there were challenges of gathering actual technology cost posed by 
the companies’ leadership, due to proprietary technology rights, privacy policies, and 
cautious in advertising actual cost without negotiating a contract. But discussions to 
estimate cost in a theoretical nature were provided by one company in order to conduct 
the thesis. Therefore, the minimum cost to buy object/shape recognition is about 
$150,000 for one facility. This includes hardware, software, and licensing. The intent of 
the study is to offer decision-makers as possible related information to what the 
commercial sector has to offer on object/shape recognition technology; so under the 
circumstances this is the best estimate provided to conduct the study. Nonetheless, the 
steps and procedures are consistent with the guidance such that it can be used to further 
exploit the actual cost with an accurate quote. 

In order to calculate the NPV, a Monte Carlo model was utilized. The model 
accounts for risk associated with future uncertainty of the 2.5% social discount for labor 
pay-raise increases, which may be approved by the U.S. government. Therefore, it turns 
out that the NPV for technology provides greater benefit than status quo. 

Step 8: Perform sensitivity analysis. A sensitivity analysis supports the CBA by 
changing the wages as a form of revenues assumptions to understand risk factors that 
investing in object/shape recognition technology inherits. Table 9 illustrates wages a 
maximum or minimum in terms of revenues. To account for the uncertainty of the 
predicted impact, this study conducted a yearly variance sensitivity analysis using the 5% 
lower bound in the NPV’s confidence interval that resulted with $56,818 and the 90% 
upper bound NPV’s confidence interval resulting in $66,536. The minimum and 
maximum values for NPV are at $53,732 and $70,400, respectively. Thus, NPV remains 
positive with the variations in wages percentage increase or decrease. Appendix C 

illustrates the sensitivity analysis that supports the CBA by changing the wages social 

discount as a form of revenues from pay raise or cut between the time value money 

discount rate for assumptions in order to understand risk factors in investing in 

object/shape recognition technology inherit. 
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Data 


Table 9. Net Present Value Results 


Initial Investment 

$ 150,000 

Cash Flow Year 1 

$ 27,282 

Growth rate ‘g' (1% Std Dev) 

2.50% 

Discount rate 'r' 

7% 

Time (yrs) 

10 


Fiscal Year 

2014 


2015 


2016 

2017 

2018 

2019 

2020 

2021 

2022 

2023 

2024 

Year 

0 


1 


2 

3 

4 

5 

6 

7 

8 

9 

10 

Growth 


3.79% 


2.28% 

2.47% 

4.03% 

2.90% 

1.61% 

2.70% 

4.56% 

2.69% 

3.78% 

Cash Flows (CF) 

-S 150,000 

5 

27,282 

$ 

27,904 

$28,594 

$ 29,747 

$30,608 

$31,102 

$31,941 

$33,397 

$34,294 

$35,591 

Discount CF FACTOR 

1 


0.93 


0.87 

0.82 

0.76 

0.71 

0.67 

0.62 

0,58 

0.54 

0.51 

Discount cash Flow 

-S 150,000 

$ 

25,497 

$ 

24,372 

$23,341 

$ 22,694 

$21,823 

$20,725 

$ 19,891 

$19,437 

$18,654 

$18,093 

Cumulative Discount CF 

B -5150,000 

-S 124,503 

-$100,131 

-$76,790 

-$ 54,096 

-$ 32,273 

-$ 11,548 

$ 8,343 

$27,780 

$46,434 

$64,527 


Net Present Value Results: 


NPV 

$ 

64,527 

Internal Rate of Return (IRR) 


15.13% 

Profitability Index (PI) 

$ 

1.43 

Pay Back (years) 


6.7 

NPV - Monte Carlo Simulation Results 

Number of Samples: 


1000 

NPV Mean 

$ 

62,265 

NPV * Standard Deviation 

$ 

3,397 

Minimum 

$ 

53,732 

Maximum 

$ 

70,400 

5% Percentile 

$ 

56,818 

90% Percentile 

$ 

66,536 


Step 9: Make a recommendation. 

The recommendation to implement object/shape recognition is viable; however it 
will solve the less than 1% of unlabeled items and not solve the 7.5 days’ worth of 
backlog items for the DLA DS facility located in Camp Pendleton. The estimated cost to 
implement one object/shape recognition system is about $150,000. The total quantified 
net economic benefit occurring from the technology’s contributions is estimated to be 
about $65,000 when applying a 7% discount rate, and a 2.5% social discount for labor 
pay-raise increases the corresponding internal rate of return results at 15.13% —making 
the investment desirable to pursue, and profitability index (PI) 1.43, indicating that for 
every dollar invested, $1.43 in benefits is accrued with a payback period 6.7 years over a 
10-year time span. Applying a 7% discount rate yields a NPV of $35,591. The payback 
period is illustrated in Figure 28. 
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A. SUMMARY 

The benefit from implementing object/shape recognition technology outweighs 
the cost to purchase it. The new technology will save money in labor costs. The 10-year 
period of revenues provided by the technology recoups the $150,000 initial investment in 
about seven years such that the internal rate of return of 15% is greater than the cost of 
capital estimated at 7%. The profitability index is $0.43 to each one dollar invested, 
suggesting that DLA DS should execute the investment and pursue it for further 
maturation. However, the scope of this thesis to research more areas of cost and benefits 
was limited to considering the new technology obtain only as an alternative to the status 
quo. In addition, the analysis used estimates of some of the elements of costs, as 
inaccurate costs of new technologies are not available. However, the CBA developed in 
this thesis can be easily adjusted to incorporate more accurate data once it becomes 
available and therefore allow a certified contracting officer to pursue further the validity 
of an object/shape recognition technology option. 
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VI. CONCLUSION 


A. INTRODUCTION 

This chapter summarizes the project—a quest for answers to the problem that 
DLA DS at Camp Pendleton faces. Provide recommendations to the four areas of 
concerns. And, brief areas of uncertainty that would require further study. 

The Lean Six Sigma study DLA DS conducted in 2008 resulted streamlining the 
item-disposal process by saving the time equivalent of 40% and concluded to conduct 
further study to exploit the operation process with technology. This thesis aimed to 
address the suggested new technology implementation option. Specifically, the thesis 
addressed the following research questions. Answering these questions was the key take 
away and culmination of the research. 

1. Is there an object/shape recognition technology to support the feasibility 
of implementing it at DLA Disposition Services? 

2. Secondary key questions to answer include the following: (1) What are 
the technology needs for DLA Disposition Services? (2) Is the 
technology mature enough for implementation? (3) Where will the 
technology and equipment be obtained? (4) How much will the 
equipment and technology cost and will it generate any tangible savings? 
(5) What is the payback period? 

To answer the research questions this study utilized a retrospective mixed mode 
analysis of DLA DS’s items disposal processing using data collected from Camp 
Pendleton, California, a technology readiness assessment (TRA) on object/shape 
recognition technology maturity level, and CBA to examine the feasibility to implement 
object/shape recognition technology. The technology readiness assessment and the CBA 
are consistent with the Circular A-94 (0MB, 1994), DOD Directive 5000.2 Interim 
(DOD, 2013a) and DOD TRA (Assistant Secretary of Defense for Research and 
Engineering [ASD(R&D)], 2011) guidance to calculate the object/shape recognition 
technology’s benefits of reducing item research time. The study is retrospective in that it 
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included and assessed only a four-month period of data and, given that DLA DS is a non¬ 
profit organization, the researcher utilized a “market comparable approach” to calculate 
the benefits, costs, and return on investment. Although DLA DS does not have an overall 
comparable commercial market force to compare revenues, this approach allowed the 
researcher to use the market comparable labor as a form of estimating revenues. 

B. PROJECT SUMMARY 

This thesis’ objective was to understand the items disposal process and assess 
object/shape recognition technology to address the issue unlabeled items poses to DLA 
DS. However, the findings generated by the analysis of DLA DS’ items disposal 
processing show three other areas of concern that contribute to days’ worth of backlog 
items. The analysis suggests that the problem is not exactly the unlabeled items. Indeed, 
employees spend excess time conducting research on items that are not labeled or 
documented on the DD 1348 form. However, unlabeled items are less than 1% of 
accepted items and it does not fully contribute to days’ worth of backlog. Three other 
area of concern contributed to days’ worth of backlog. First, improperly filled DD 1348 
provides for additional time spent conducting research in order to fill the DD 1348 form 
for further processing. Second, the receipt control numbers (RCNs) are not limited to the 
amount of items that can arrive in a pallet or tri-box. A RCN can have 1 or 50 items. This 
creates the potential to accept an average of 425 items in one day and DLA DS at Camp 
Pendleton capacity to process items is limited to 300 items with six-employee crew. 
Therefore, third, DLA DS at Camp Pendleton accepts more items than what it can 
process, which results in having an average of over 2,000 items in the backlog stowage 
area. The facility processes an average 300 items a day and had an average 47 RCN 
equaling to an average of 2,259 items or 7.5 days’ worth of backlog items stored for later 
processing. 

This thesis examined the alternative of addressing the unlabeled items issues with 
the use of new object/shape recognition technology. The DLA DS technological 
requirement includes item recognition and disposal information feedback, and the ability 
to take pictures to use on DLA email system to communicate and upload the picture to 
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liquidations website. This thesis assessed the readiness level of existing object/shape 
recognition technology by analyzing three available such technologies: 

First, technologies that are ready with DLA DS’s specifications and operate in 
DLA DS similar environment have not been created, therefore none meet the required 
threshold of TRL 7 or above. 

Cognex designed machines specific to customers’ requirement and the company 
does not have a machine ready for DLA DS to operate straight out of the box, therefore 
Cognex TRL is 2. To implement Cognex’s technology to DLA DS it will require further 
maturation of technology costing excess time and funding making Cognex technology 
less desirable to be adopted by DLA DS. 

Imaginestics technology is an application for mobile devices and requires Internet 
to access the database for feedback. Lockheed Martin is utilizing Imaginestics’ 
technology. The DOD funded Imaginestics’ technology. Furthermore, Imaginestics has 
been approved for information assurance (lA) for another technology under the Navy and 
Office of the Secretary of Defense (OSD), therefore the company understand the DOD 
acquisition and lA approval process. However, Imaginestics object/shape recognition 
technology it ranks at TRL 5, but Imaginestics technology rates higher TRL amongst the 
three companies making it more desirable to pursue further maturation of Imaginestics 
technology. Investing in Imaginestics technology will cost less time and funding 
compared to the other two companies. 

RMD’s automated digital recognition technology (ADART) was artificial 
intelligence tool designed for the Marine Corps to capture and recognize mechanics’ 
repair process for identification and feedback. However, the funding to further mature the 
technology was stalled and the machine is still in a prototype raking in TRL 2. To 
implement RMD’s technology to DLA DS it will require further maturation of 
technology costing excess time and funding making RMD technology less desirable to be 
adopted by DLA DS. 

The benefit from implementing object/shape recognition technology outweighs 
the cost to purchase it. The new technology will save money in labor costs for the 
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unlabeled items that are less than 1% of accepted items. The 10-year period of revenues 
provided by the technology recoups the $150,000 initial investment in about seven years 
such that the internal rate of return of 15%, which is greater than the cost of capital 
estimated at 7%. The profitability index is $0.43 to each one dollar invested, suggesting 
that DLA DS should execute the investment and pursue it for further maturation. 
However, implementing object/shape recognition technology does not fully contribute the 
reduction of days’ worth of backlog. Therefore, the results from the queuing theory 
suggests that with nine employees the DLA DS at Camp Pendleton will keep up with 
demand by processing all received items and will progressively reduce the 7.5 days’ 
worth of backlog. 

One of the main conclusions resulted from this thesis effort is that researching 
items is not the only area of concern with regard to process inefficiency. Rather, this 
thesis identified three other areas of concern suggested by the data analysis and 
observation. Therefore, further recommendations were provided to take care of the other 
three problems. 

C. RECOMMENDATIONS 

Given the main findings identified by this thesis analysis, the recommendation to 
implement object/shape recognition is viable as far as generating a positive return to 
investment; however it only address less than 1% of items and not the 7.5 days’ worth of 
backlog items for the DLA DS facility located in Camp Pendleton. Investing in 
Imaginestics’ technology for further maturation will ease the disposal process for the 1% 
of unlabeled items. 

Although this study recommends object/shape recognition to alleviate the 1% of 
accepted items, DLA DS must tackle the other three areas of concern with: 

1. Making mandatory to fill the DD 1348 on DLA DS website, and 

2. Limiting the amount of items on pallets to match the facility’s processing 
capacity, and 

3. Increasing the Camp Pendleton facility to nine employees, or 
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4. Adding a night shift such that the shift only process items without 
accepting or conducting mundane tasks, or 

5. Once a month stop accepting items for disposal process for a full week in 
order to focus the effort to dispose the backlog items, or 

6. Fill the gap with technology by implement full automation with optical 
sorting and data mining that included sensors, laser, object/shape 
recognition technology on conveyor belt system and data analytics to 
improve materiel’s opportunity to be sold as commodity or liquidations 
and improve operational efficiency; which would be consistent with DLA 
Director’s “Big Idea” and strategic technological goals to accomplish. 

The technological innovation disrupts status quo by ensuring a plan of increased 
capacity for the long run and perhaps reduces some of the over $60 million of labor cost 
that account for about 35% of DLA’s budget. 

D. RECOMMENDATION FOR FUTURE RESEARCH 

Areas of uncertainty remain for future study. This study was not able to capture 
time spent in every step of the item disposal process, or observe whether the backlog is a 
cyclical trend due to retrograding units from deployments, cyclical due to time of the 
year, and value added from items that are scrapped rather than reaping on the opportunity 
to be sold and increase revenues. For example if the technology increases the opportunity 
to send more items for liquidations the opportunity would be in the benefit of increasing 
revenues to DLA. Additionally, the scope of the thesis narrowed the study the Camp 
Pendleton DLA DS facility, such that the results may not be indicative of all DLA DS. 
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APPENDIX A. DISCLAIMER 


The following disclaimer was adopted from the Department of Energy (DOE, 
2012) because it is relevant to this study: 

This report was prepared as an account of work sponsored by an agency of 
the United States Government. Neither the United States Government nor 
any agency thereof, nor any of their employees, makes any warranty, 
express or implied, or assumes any legal liability or responsibility for the 
accuracy, completeness, or usefulness of any information, apparatus, 
product, or process disclosed, or represents that its use would not infringe 
privately owned rights. Reference therein to any specific commercial 
product, process, or service by trade name, trademark, manufacturer, or 
otherwise does not necessarily constitute or imply its endorsement, 
recommendation, or favoring by the United States Government or any 
agency thereof. The view and opinions of authors expressed therein do not 
necessarily state or reflect those of the United States Government or any 
agency thereof (p. iii). 
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APPENDIX B. NORMALIZED DATA 


Jul 

through.... 

7/31/14 

sorted by 
Region, Site 

8i Date 

Point in 

Time 

RCN Open 

Point in 

Time 

RCN AGE 

RCN to 

Induct 

(RCNs) 

Point in 

Time 

Backlog 

Ending 

(Lines) 

Receipts 
(Lines + 
Batches) 

Point in Time 

ADD New DAILY 

Data to bottom 
(Paste Values), 
then Sort by 

Region, Site, 

Date 

Backlog Ratio 

inventory 



Estimete 

processing 

Results 

Date 

Region 

Site iD 

Site 

Count 

Oldest RCN 

Days Avg 

Backlog 

Count 

Receipts 

Ratio 

7/1/14 West 

RCCP 

DLA DS PEND 

30 

12 

6.6 

1,082 

466 

232% 

7/2/14 West 

RCCP 

OLA DSPENOU 

30 

12 

8.6 

1,104 

256 

431% 

7/3/14 West 

RCCP 

DLA DS PENDLI 

30 

12 

8.6 


1,703 

375 

454% 

7/4/14 West 

RCCP 

DLA DS PENDLI 

30 

12 

- 


b 


7/5/14 West 

RCCP 

DLA DS PENDLI 

30 

12 



b 

7/6/14 West 

RCCP 

DLA DS PENDLI 

28 

12 

- 

1.703 

0 

7/7/14 West 

RCCP 

DLA DS PENI 

31 

13 

9.9 

2,320 

231 

1004% 

7/8/14 West 

RCCP 

DLA DS PEN( 

31 

14 

10.0 

2,417 

357 

677% 

7/9/14 West 

RCCP 

DLA DS PENI 

34 

13 

7.9 

2,507 

311 

806% 

7/10/14 West 

RCCP 

DLA DS PENI 

37 

14 

3.6 

2,530 

298 

849% 

7/11/14 West 

RCCP 

DLA DS PENI 

37 

14 

8.5 


2,538 

273 

930% 

7/12/14 West 

RCCP 

DLA DS PENI 

37 

14 

- 

- 

b 


7/13/14 West 

RCCP 

DLA DS PENI 

40 

17 

-- 

2,538 

b 


7/14/14 West 

RCCP 

DLA DSPENI 

44 

15 

14.3 

2,353 

351 

670% 

7/15/14 West 

RCCP 

DLA DSPENI 

47 

16 

12.5 

2,108 

333 

633% 

7/16/14 West 

RCCP 

DLA DS PENI 

51 

17 

13.5 

2,055 

303 

678% 

7/17/14 West 

RCCP 

DLA DSPENI 

51 

18 

12.5 

2,099 

301 

697% 

7/18/14 West 

RCCP 

DLA DS PENI 

51 

18 

10.5 


2,544 

232 

1097% 

7/19/14 West 

RCCP 

DLA DS PENI 

51 

18 

16.1 


401 

0% 

7/20/14 West 

RCCP 

DLA DS PENI 

45 

18 

- 

2,544 

b 


7/21/14 West 

RCCP 

DLA DSPENI 

47 

19 

15.4 

2,446 

157 

1558% 

7/22/14 West 

RCCP 

DLA DSPENI 

54 

16 

13.4 

2,431 

213 

1141% 

7/23/14 West 

RCCP 

DLA DS PENI 

59 

17 

6.4 

2,410 

107 

2252% 

7/24/14 West 

RCCP 

DLA DS PENI 

51 

18 

4.9 

2,273 

230 

988% 

7/25/14 West 

RCCP 

DLA DS PENI 

51 

18 

6.4 

2,456 

401 

612% 

7/26/14 West 

RCCP 

DLA DSPENI 

51 

18 

-- 

- 

b 


7/27/14 West 

RCCP 

DLA DSPENI 

52 

21 

- 

2,456 

b 


7/28/14 West 

RCCP 

DLA DS PENI 

47 

22 

10.2 

2,313 

508 

455% 

7/29/14 West 

RCCP 

DLA DS PENI 

49 

23 

13.7 

2,383 

432 

552% 

7/30/14 West 

RCCP 

DLA DS PENI 

53 

24 

6.9 

2,601 

520 

500% 

7/31/14 West 

RCCP 

DLA DS PENI 

62 

25 

13.7 

3,072 

140 

2194% 


Figure 29. Normalized Data Treated with Excel’s Default Setting “Show empty data 

as zero” 


99 
































































































THIS PAGE INTENTIONALLY LEET BLANK 


100 



APPENDIX C. NPV SENSITIVITY ANALYSIS TABLE 



Sensitivity Analysis - NPV 


Discount Rate V 



5% 

6% 

7% 

8% 

9% 

10% 

11% 

12% 

13% 

14% 

15% 


2% 

$ 

78,844 

$ 

67,792 

$ 

57,520 

$ 

47,963 

$ 

39,058 

$ 30,752 

$ 22,995 

$ 

15,743 

$ 

8,954 

$ 

2,594 

-$ 3,373 


3% 

S 

88,652 

$ 

76,954 

$ 

66,088 

$ 

55,983 

$ 

46,574 

$ 37,802 

$ 29,616 

$ 

21,966 

$ 

14,810 

$ 

8,108 

$ 1,825 


4% 

S 

98,971 

$ 

86,588 

$ 

75,094 

$ 

64,409 

$ 

54,467 

$ 45,203 

$ 36,562 

$ 

28,493 

$ 

20,948 

$ 

13,887 

$ 7,270 


5% 

$ 

109,828 

$ 

96,721 

$ 

84,560 

$ 

73,264 

$ 

62,757 

$ 52,973 

$ 43,852 

$ 

35,338 

$ 

27,383 

$ 

19,941 

$ 12,972 


6% 

S 

121,252 

$ 

107,377 

$ 

94,512 

$ 

82,567 

S 

71,463 

$ 61,129 

$ 51,500 

$ 

42,518 

$ 

34,129 

$ 

26,286 

$ 18,945 


7% 

$ 

133,269 

$ 

118,583 

$ 

104,972 

$ 

92,341 

$ 

80,606 

$ 69,691 

$ 59,526 

$ 

50,048 

$ 

41,201 

$ 

32,935 

$ 25,202 


8% 

$ 

145,912 

$ 

130,367 

$ 

115,967 

$ 

102,611 

$ 

90,209 

$ 78,679 

$ 67,947 

$ 

57,946 

$ 

48,616 

$ 

39,902 

$ 31,755 


9% 

$ 

159,211 

$ 

142,757 

$ 

127,523 

$ 

113,401 

$ 

100,293 

$ 88,114 

$ 76,783 

$ 

66,230 

$ 

56,390 

$ 

47,204 

S 38,620 


10% 

$ 

173,200 

$ 

155,785 

$ 

139,668 

$ 

124,736 

$ 

110,884 

$ 98,018 

$ 86,055 

$ 

74,919 

$ 

64,540 

$ 

54,856 

$ 45,812 


11% 

S 

187,914 

$ 

169,482 

$ 

152,433 

$ 

136,644 

$ 

122,004 

$ 108,414 

$ 95,784 

$ 

84,032 

$ 

73,084 

$ 

62,876 

$ 53,345 


12% 

$ 

203,388 

$ 

183,881 

$ 

165,847 

$ 

149,153 

$ 

133,682 

$ 119,326 

$ 105,991 

$ 

93,589 

$ 

82,042 

$ 

71,280 

$ 61,237 


Figure 30. A Sensitivity Analysis of Wages between Social Discount Rate (Growth “g”) and Discount Rate “r” 
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Figure 30 illustrates the sensitivity analysis that supports the CBA by changing 
the wages social discount as a form of revenues from pay raise or cut between the time 
value money discount rate for assumptions in order to understand risk factors in investing 
that object/shape recognition technology inherit. 
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